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Abstract 

In this paper, the problem of designing network codes that are both communicationally and computationally 



' efficient over packet line networks with worst-case schedules is considered. In this context, random linear network 

codes (dense codes) are asymptotically capacity-achieving, but require highly complex coding operations. To reduce 
I the coding complexity, Maymounkov et al. proposed chunked codes (CC). Chunked codes operate by splitting 

. the message (a collection of packets) into non-overlapping chunks and send a randomly chosen chunk at each 

in 



transmission time by a dense code. The complexity, that is linear in the chunk size, is thus reduced compared to 



lO . dense codes. In this paper, the existing analysis of CC is revised, and tighter bounds on the performance of CC are 

o 

derived. As a result, we prove that (i) CC with sufficiently large chunks are asymptotically capacity-achieving, but 
with a slower speed of convergence compared to dense codes; and (ii) CC with relatively smaller chunks approach 
' the capacity with an arbitrarily small but non-zero constant gap. To improve the speed of convergence of CC, 

, while maintaining their advantage in reducing the computational complexity, we propose and analyze a new CC 

scheme with overlapping chunks, referred to as overlapped chunked codes (OCC). We prove that for smaller chunks, 
which are advantageous due to lower computational complexity, OCC with larger overlaps provide a better tradeoff 
between the speed of convergence and the message or packet error rate. This implies that for smaller chunks, and 
with the same computational complexity, OCC outperform CC in terms of the speed of approaching the capacity 
for sufficiently small target error rate. In fact, we design linear-time OCC with very small chunks (constant in 
the message size) that are both computationally and communicationally efficient, and that outperform linear-time 
CC. Finite-length simulation results consistent with the asymptotic analytical results are also presented. Both the 
analytical and simulation results suggest great potential for the application of OCC for multimedia transmission 
over packet networks. 

preliminary version of this work was partly presented in ITW 2010, Cairo, Egypt, January 2010, and has been accepted in part for 
presentation at ISIT 2011, Saint-Petersburg, Russia, August 2011. 
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Index Terms 
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I. Introduction 

THERE has recently been a surge of interest in application of network coding for large-scale file 
sharing over packet networks [ID. Network coding has been shown to generally reduce the expected 
file downloading time for various probabilistic and deterministic models of the flow transmission schedules 
1121, [Ell. In practice, however, not only the network nodes might be blind to the schedule, but also the 
accurate modeling of the schedule might be too complex and/or infeasible [|4l-[[6l. The problem of a 
practical code design is therefore to achieve the capacity of the network with high probability under any 
arbitrary schedule unknown at the network nodes. 

Random linear network codes (a.k.a. dense codes) are known to achieve the capacity in this setting 
asymptotically (when the message length tends to infinity), while having linear coding costs (the encod- 
ing/decoding algorithms at each node require a number of packet operations per message packet linear 
in the message length) [|3, [[HI. The rather high coding cost, however, impedes the application of dense 
codes for the transmission of large files. One would thus be interested in devising coding schemes with 
relatively low complexity. 

To overcome the computational inefficiency of dense codes, Maymuonkov et al. proposed chunked 
codes (CC) in [[8][. These codes operate by dividing the original message into non-overlapping chunks. 
Each node then randomly chooses a chunk at any time instant and transmits it by using a dense code. 
Thus, CC require less complex coding operations as they apply coding on smaller chunks rather than the 
original message. (The coding costs of such codes are linear in the size of the chunks.) This advantage 
of CC, however, comes at the cost of lower speed of convergence or higher message or packet error rate 
compared to dense codes. 

In [[HI, it has been shown that CC asymptotically achieve the capacity so long as the size of chunks 
(a.k.a. aperture size) is bounded below. This lower bound has been shown to be an increasing function of 
the message length. Thus the aperture size cannot be reduced down to a constant in the message length. 
The coding algorithms, therefore, cannot be performed in linear time (with constant costs in the message 



length). This may hamper the use of such codes in practical applications with severe computational 
resource limitations. 

The need for coding schemes with smaller coding costs motivated the authors in [[8l to also analyze 
CC with smaller apertures. They showed that a CC with chunks of smaller sizes (down to some constant 
in the message length) approaches the capacity with an arbitrarily small but non-zero constant gap. 

A. Main Contributions 

Targeting the design of codes with better tradeoff between the computational complexity, on one 
hand, and the speed of convergence and the message or packet error rate, on the other hand, the main 
contributions of this paper are as follows: 

• We provide a more comprehensive analysis of dense codes, compared to that of This will then 
serve as the basic framework for the analysis of CC and OCC. 

• We revise the analysis of CC in [[8l, and prove that CC with any aperture size provide a better tradeoff 
between the computational complexity, on one hand, and the speed of convergence and the message 
or packet error rate, on the other hand, in comparison with what was previously thought, based on 
the results of jSl. 

• We propose chunked codes with overlapping chunks, referred to as overlapped chunked codes (OCC), 
and show that (i) for sufficiently large apertures, OCC (OCC with larger overlaps) achieve the capacity, 
but with a slower speed of convergence compared to CC (OCC with smaller overlaps), for any given 
message error rate; (ii) for smaller apertures, OCC (OCC with larger overlaps) approach the capacity 
with an arbitrarily small but non-zero constant gap with a larger speed of convergence when compared 
to CC (OCC with smaller overlaps), for sufficiently small given message or packet error rate. 

• The result of (ii) leads to the design of linear-time network codes (with very small apertures of 
constant size with respect to the message length), which perform better than the existing codes in 
the literature with the same computational complexity over networks with arbitrary schedules. 

• As part of the machinery used in the analysis, we generalize a recently proposed conjecture in [|9] 
on the rank property of a special class of random matrices with overlapping bands to two classes of 
more general banded random matrices |3 

• We also demonstrate the advantage of finite-length OCC (OCC with larger overlaps) over CC (OCC 
with smaller overlaps) through extensive simulation results. For example, our results show that when 

'The validity of our conjecture is supported via simulations, but a formal proof is still unknown. 
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compared to a CC with similar coding costs, the application of OCC can decrease the downloading 
time of a 1MB file from a file server 4 hops away by about 17% — 30%. 

B. Related Work 

There are a number of variations of chunk-based codes in the literature of efficient network codes as 
well as efficient erasure-correcting codes over a single binary erasure channel (BEC). To the best of our 
knowledge, however, none of these codes have been provably shown to perform better than that in [|8] 
over arbitrary schedules unknown at the network nodes. 

Focusing on the design of computationally efficient codes over BEC, in [|9l, Studholme and Blake 
propose "windowed erasure codes." These codes have a similar structure to the chunked codes of [8], 
except that every two contiguous chunks overlap in all but one packet. From the results of [9|, it can be 
concluded that, compared to non-overlapping chunks, the application of overlapping chunks provides a 
significant improvement in the speed of convergence to the capacity and/or in the probability of decoding 
failure. This advantage arises from the large size of the overlap between the chunks. However, the larger 
is the overlap size, the larger will be the number of chunks. Having a large number of chunks, regardless 
of whether the chunks overlap or not, hampers the application of such codes over a longer network of 
erasure links (as shown in [[81). To remedy this situation, we reduce the overlap size in our version of 
chunked codes with overlapping chunks. We in fact demonstrate that with the right balance between the 
overlap size and the number of chunks, OCC can be a viable choice for information transmission over 
packet networks. 

The idea of overlapping chunks has also been proposed by Silva et al. in ifTOl . independently. Unlike our 
contiguous overlapping scheme, the overlapping scheme of [|TOl has a grid structure. Also, no theoretical 
result is presented in [TO]; and the simulation results are only on the application of such coding schemes 
over a single BEC, not a (longer) line network. In IfTTl . Li et al. propose a randomized overlapping scheme 
and provide a finite-length analysis of such codes over the BEC. Our preliminary simulations demonstrate 
that the codes proposed in [fTTI do not perform well over longer line networks. Moreover, the analysis in 
IfTTl does not seem to be generalizable to line networks. In this work, we provide an asymptotic analysis 
of overlapped chunked codes (with contiguous overlaps) over line networks. The proposed codes are 
superior to those of [[TOl and [fTD in the underlying setting. The structure of overlapping schemes in [[TOl 
and IfTTTl implies that for a low-complexity decoding algorithm, the chunks need to be decoded one at a 
time. In our work, however, the decoding can be performed on the set of all the chunks together while 
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Fig. 1. A line network of length /. 

preserving the low complexity of the decoding algorithm. This thus results in a better tradeoff between 
the speed of convergence to the capacity and the message or packet error rate. In fact, by performing the 
decoding algorithm on the set of all the chunks simultaneously, for successful decoding, no chunk needs 
to be recoverable in isolation. Thus a smaller number of successful packet transmissions is sufficient to 
ensure successful decoding for a given message/packet error rate. This, however, may come at the cost 
of increasing the memory requirements. 

The rest of the existing literature on chunk-based codes consider problem settings that are different 
from ours. In particular, in [fT2l - [fT4l . some knowledge about the schedule is available at the network 
nodes. In addition, in [[T5l - [fT71 , a probabilistic model for the schedule is assumed. Unlike these, in this 
work, we assume "arbitrary" schedules which are "unknown" at network nodes. 

C. Organization 

The remaining of the paper is organized as follows: Section HI] presents the problem formulation and 
definitions. In Section [nil we study the capacity-achieving codes, i.e., dense codes, CC and OCC with 
large chunk sizes. Section JV] contains the analysis of the capacity-approaching codes, i.e., CC and OCC 
with small chunks. Section IVl covers the simulation results, and Section |VI] concludes the paper. 

II. Problem Formulation and Definitions 

A. Network Scenario 

In this paper, we focus on the information flow problem over line networks. The results however can 
be generalized to more general network scenarios over arbitrary schedules by a union bound analysis. 

Line Networks: The collection of / links connecting / + 1 nodes {vi : < i < 1} in tandem is called 
a line network of length I, i.e., for each < i < I, there is a directed link (n, v) between the two nodes 
u = Vi and v = vi^i (see Figure [I]). The node u (node v) is said to be transmitting {receiving) over the 
link {u,v). We consider a unicast problem as follows. The node s = vq, called source, generates a vector 
of messages; the node t = vu called sink {receiver), demands the vector of messages generated at node s. 
The rest of the nodes {vi : < i < /} in the network are called interior, and are responsible for relaying 
the messages from the source to the sink. 
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Field and Vector Space: Suppose that node s is given a set of /c messages {xi : I <i < k}. The 
messages are each drawn from an L-dimensional vector space F (= F^) over a finite field F. The index 
i is called the label of Xi. We call a vector y E J-' a packet. We denote the set of labels of the message 
packets by A^^j^ 

Schedule over the Network: Suppose that following a certain timing schedule, node u transmits a packet 
over the link (m, v) in any opportunity that it gets. The links may be lossy, i.e., a packet which is sent by a 
node u may not reach the receiving node v. In this case, the packet is called an erased packet, otherwise, 
it is referred to as a successful packet. We assume that the erasure rates of different links may be different 
and may also be time-varying. In addition, the links are assumed to have arbitrary time-varying delays. 
Thus, the successful packets might be re-ordered at the receiving end of a link. We also assume that 
the network nodes have infinite memory, i.e., no received packet is discarded over time by any receiving 
node. Moreover, we consider a scenario where network nodes are blind to the schedule of transmission 
of successful packets (called schedule), i.e., no feedback information is available. 

Graphical Representation of a Schedule: We use a digraph, called trellis connectivity graph (or trellis 
for brevity), to represent a given schedule (see Figure |2). The trellis only represents the successful 
transmissions through the network. The node set of the trellis includes the nodes Ur for each network 
node u so that a successful packet either departs from or arrives at the node u at time r, and two extra 
nodes sq and t^oU The edge set of the trellis consists of two groups of directed edges specified as follows; 
(i) traffic edges: every 2-tuple [ur.Vr'), r < r', representing an in-edge (out-edge) of node v (node u), 
for every pair of nodes u^. and Vr', if there is at least one packet sent by node u at time r and received 
by node v at time r', and (ii) memory edges: every 2-tuple (ur.Ur'), r < r', if, for all r" E {t,t'), there 
is no node Ur". 

Without loss of generality, the traffic edges are assumed to have unit capacity, i.e., only one packet is 
successfully sent over a traffic edge, as parallel traffic edges are allowed. Also, the memory edges are 
assumed to have infinite capacity, i.e., at any given time, a network node has access to all its successfully 
received packets. 

^The finite field F is defined based on two operations "addition" and "multiplication," respectively represented by symbols "+" and "•". 
We assume that both operations have the same cost, as one field operation. We also define two types of operations in a vector space J^: (i) 
y + z, for y,z £ iF, is taken to be symbol-wise with respect to operator + in F, and requires one packet operation, and (ii) \y, for A G F 
and y £ J-, symbol-wise with respect to operator . in F, and also requires one packet operation. 

^The node so represents the node s at time zero when all the message packets are available, and the node too represents the node f at a 
time after which there is no more coded packet to arrive. 
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Fig. 2. A trellis for a schedule over a line network of length I — 3. The edges (so, Sri), (sn , ST2), and (u^/ , u^/ ) are examples of memory 
edges and the edge (5^ , u^^ ) is an example of a traffic edge. 



Capacity of Schedule: The maximum number of message packets that can be successfully sent through 
a network with a given schedule is called the capacity of network under the schedule or simply the 
capacity of the schedule. 

Modeling a schedule as a flow network, the capacity of the schedule equals the maximum flow between 
the nodes sq and too- By the max-flow min-cut theorem, the capacity of a schedule equals the minimum 
of the sum of the capacities of the cutset edges among all the cutsets in the trellis. This quantity is called 
the min-cut capacity. 

The min-cut capacity is achievable if the network nodes are able to (properly) process their received 
packets and generate new packets to be sent. When processing is allowed at the nodes, the information 
flow scheme is called network coding, and the min-cut capacity is therefore often referred to as the network 
coding capacity. 

When the network nodes are only allowed to generate coded packets by linearly combining their received 
packets, the network coding scheme is called linear. When network codes are restricted to be linear, the 
maximum number of message packets that can be sent through a network with a given schedule is called 
the linear coding capacity. We focus on linear network codes in this work and for brevity, hereafter, we 
refer to the linear coding capacity as the capacity. It should be noted that the linear coding capacity is 
equal to the routing capacity for the unicast scenario if the schedule is known at the network nodes. This 
capacity is equal to the number of (traffic) edge-disjoint paths between the source and the sink. In the 
absence of the knowledge of the schedule at the network nodes, which is the case in this work, however, 
routing does not achieve this capacity. 

In the following, to analyze schedules of a given capacity n, we shall focus on schedules in which 
there are n edge-disjoint paths in the trellis starting from sq and ending at too- 
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Fig. 3. An example of a network of length 2 under a schedule of capacity 4 with exactly 4 sent and/or received packets at any network 
node. 

Let (Ou^) be the set of in-edges (out-edges) of node u prior to time r. Cleariy X.^ = Ot^ = 0, for 
all T E [0, oo). We label the edges in X^^ (Ou^), so that the first in-edge (out-edge) at node u has label 1, 
the second has label 2, and so forth. With a slight abuse of notation, we also use the notation X^^ (Cu^) 
for the set of labels of edges in X^^ (Our)- 

In this work, we are interested in worst-case schedules. We thus assume a given network under a 
schedule of capacity n, so that (i) for every interior node u, |X„^| = \Ou^ \ = n, i.e., precisely n packets 
are sent and received by node u, and (ii) for any time r, \Ou^\ < |X„^|, i.e., the number of successful 
transmissions at each interior node u at any time r does not exceed the number of successful receptions 
at node u. A schedule satisfying both conditions is shown in Figure |3l 

Condition (i) corresponds to having the minimum number of successful transmissions that can support 
a schedule of capacity n. Condition (ii) ensures that under the constraint of Condition (i), the schedule 
has n edge-disjoint paths. 

Coding over the Schedule: In this work, we restrict the codes to be linear. A coded packet to 
be sent over the out-edge e of node (or Sr) is generated by Xljex„ -^e.il/i (or XljeA^ where 
Ae,j, l^e,j G IF, for all i G X^^, and all j G A^^, and E J-' is the packet received along the in-edge i of 
node u, for all i E X„^. For all a; G A^, :e is estimated by J2ieXt ^x,iyi, where Xx^i E F, for all i E Xt^, 
and G X is the packet received along the in-edge i of node t. 

We refer to a method of generating codes as a coding scheme, and associated with a coding scheme is 
a class of codes generated by the coding scheme. 

We say that a code in a class C of codes over a vector space X fails over a given network with a 
schedule of capacity n if node t fails to recover all the k message packets {x E from the set of n 
packets {v^, : e E 1-t^}\ otherwise, the code is said to succeed. The ratio k/n h referred to as the code 
rate. The probability that a randomly selected code in C fails is referred to as the probability of failure 
of the class C of codes, and it is denoted by ek^n- 

Let kn be a function of n (kn < n), denoting the number of message packets at the source node, and let 
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efc„,n (0 < Cfc^ „ < 1) be a function of n (and A;„). We say that a coding scheme achieves the capacity or 
approaches the capacity with gap A over a given network with any schedule of capacity n, for < A < 1, 
if there exists a sequence of codes of rates {kn/n} so that the sequence of failure probabilities {ekn,n} 
goes to e, for some < e < 1, arbitrarily small, and the sequence of rates goes to 1 or 1/(1 + A), as n 
goes to infinity. 

Let fcnmax bc the largest function of n, so that for a given < e < 1, with probability of failure no 
larger than e, a given coding scheme achieves/approaches the capacity over a network of any schedule 
of a given capacity n. For any given n, the larger is the ratio /cnmax/'^> the larger is said to be the speed 
of convergence of a coding scheme. That is, for a given n, with a given probability of failure, a coding 
scheme with a larger speed of convergence is able to transmit more message packets over any schedule 
of capacity n. 

Complexity of Codes: The number of packet operations for applying the encoding functions divided by 
k-l is called the encoding cost, (i.e., we add up the number of packet operations needed to generate all the 
coded packets at all the non-sink nodes, and normalize it by the number of message packets multiplied 
by the number of links). The number of packet operations for applying the decoding functions divided 
by k is called the decoding costc. 

B. Problem Formulation 

Suppose a network of length I under an arbitrary schedule with capacity n. For any given coding 
scheme, our goal is to derive tight upper bounds on (i) the number of message packets {k) drawn from 
a vector space F (over at node s, as a function of n, I and/or A, in the asymptotic regime (i.e., as 
n goes to infinity), so that the coding scheme over the vector space F fails to achieve or approach the 
capacity with a given gap < A < 1, with probability no larger than < e < 1, and (ii) the encoding 
and decoding costs. 

''The coding costs exclude field operations that are independent of the size of the packets (i.e., L). The reason is that L is usually very 
large in practice and the computations dealing with packet operations dominate the computational cost. 

^We restrict the finite field F to the binary field F2 (i.e., each message packet is a stream of L bits). The reason for this restriction is 
two-fold: (i) to have the lowest computational complexity, and (ii) to consider the case with the lowest speed of convergence (the larger 
is the field, the larger would be the speed of convergence of any coding scheme considered in this paper, but at the cost of increasing the 
computational complexity). 
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III. Capacity-Achieving Codes 

We start with the analysis of random linear codes (a.k.a. dense code^ over line networks of length /, 
with arbitrary schedules of a given capacity n, as n goes to infinity. 

We address the following issues: (i) a dense code over a vector space F2 (for any integer L) is capacity - 
achieving, and (ii) the encoding and decoding costs are each 0{n). 

A. Dense Codes 

Encoding: For every out-edge e of node n^, = Yliex^ ^e^iVi if u is interior, and = Yl,jeMs /^e.i^i 
if u is the source, where for all i E \ ^ur^ ^e,i is 0, and for all i E Zu^, and all j E Aig, \e,i and /iej 
are symbols independently and uniformly drawn from F2. 

Since every coded packet is a result of linearly combining message packets, every coded packet y^ 
over an out-edge e of node Ur, for an interior node u, can be written as 



Ve = Yl ^^'^ Yl 
jeMs ieiuoo 



Xj 



where fiej := J2ieXu and /ijj's can be defined recursively. 

The vector Ag of n elements {\e,i ■ i G ^uo^} i^ called the local encoding vector of packet y^. The 
vector /Xg of k elements {fiej ■ j ^ -Ms} is called the global encodingvector of packet y^. The global 
encoding vector of a packet is sent along with the packet in its headeri_ 

Let Mu be a subset of X„^. We say that a collection {y^ '■ ^ & A/^,} of packets at a receiving node v 
is innovative if their global encoding vectors are linearly independent. We refer to such a collection of 
packets by its set of in-edges Af^. 

Decoding: For every x E Ai, node t provides an estimate x = J2ieit ^x,iyi by solving the system 
of linear equations {y^ = Xljex^ l^ej^j '■ ^ ^ '^00} for ^ packets {xj : j E Ms}- 

Hn a random linear coding scheme as explained later in detail, each packet sent by a node is a random linear combination of all previously 
received packets. Thus the number of non-zero coefficients in linear combinations is rather large, resulting in dense linear combinations. 

dense code is set up to never transmit a coded packet with all-zero global encoding vector. Whenever such a packet is generated by a 
node, it will be discarded and a new packet will be generated till the resulting packet has a global encoding vector with at least one non-zero 
entry. 
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This system is uniquely solvable and for all j G A^s, Xj equals Xj if there is an innovative collection 
Mt at node t such that \J\ft\ = k. Thus, a dense code succeeds if node t receives a collection of k packets 
which form an innovative collection. 

Let be a matrix of size n x k whose rows are global encoding vectors of packets received by a 
node V, i.e., for all e G X^^, and for all j G Ms, iQv)e,j = /^ej- We call the decoding matrix at node 
f ; if node v needs to recover the message packets, it has to solve a system of linear equations whose 
unknowns, known constants and coefficients are the message packets, the received packets at node v and 
their global encoding vectors, respectively. This can be done, e.g., using Gauss-Jordan elimination ifTSl . 

Analysis [Outline]: The packets over the first link are random linear combinations of the message 
packets and their global encoding vectors' entries are all independently uniformly distributed (i.u.d.) 
Bernoulli random variables. Suppose that the receiving node of the first link in the network is to recover 
the k message packets after receiving n packets. This can be done so long as there exist k packets with 
linearly independent global encoding vectors. Now, a question is to find the probability of existence of 
such a collection of packets at the node. A lower bound on the probability that a set of n vectors of 
length k whose entries are i.u.d. Bernoulli random variables includes k linearly independent vectors is 
well-known, see, e.g., EH Proposition 2]. 

The packets over any lower link are also each a random linear combination of the packets previously 
received over the upper link. The entries of the global encoding vectors of these packets are uniformly 
distributed Bernoulli random variables but not necessarily independent. To the best of our knowledge, 
there is no non-trivial lower bound on the probability that a set of n such vectors of length k (whose 
entries are all uniform Bernoulli but not all independent) includes k linearly independent vectors. 

In the following, we show that the entries of the global encoding vectors of any collection of packets 
at any given node are i.u.d. Bernoulli random variables so long as the packets' local encoding vectors 
are linearly independent. This implies that the global encoding vector of any packet over any lower link 
can be written as a linear combination of those global encoding vectors over the upper link whose entries 
are all i.u.d. The main idea in our analysis is therefore to track the distribution of the size of a maximal 
collection of linearly independent local encoding vectors at each node. 

The number of packets with linearly independent local encoding vectors in general depends on the 
schedule. In this work, however, the schedule is arbitrary. We hence study the two extremal categories of 
worst-case schedules in which the probability that the local encoding vector of any packet sent by any 
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network node is linearly independent to those of the packets sent earlier by the node is the "largest" or 
the "smallest" possible, respectively. 

The first category of worst-case schedules consists of those in which any network node successfully 
transmits one and only one packet between any two contiguous arrivals. In this case, the smallest number 
of received packets are available to contribute in generating any packet to be transmitted (see Figure HJ; 
and thus the local encoding vector of any given packet is linearly dependent on those of the rest of the 
packets with the largest probability. We refer to such schedules as one-in-one-out. 

The second category of worst-case schedules consists of those in which any network node transmits 
its first successful packet after receiving all the n packets over the upper link (see Figure |5). We refer to 
such schedules as all-in-all- out. 

The local encoding vectors not only depend on the schedule but also are random. The number of linearly 
independent local encoding vectors over any given link is therefore a random variable. In the following, 
we derive probabilistic lower bounds on the value of this random variable for each type of worst-case 
schedules of interest. Taking a union bound over the number of links, we then provide a lower bound on 
the number of packets with i.u.d. Bernoulli entries over the last link. Finally, by applying a lower bound 
on the probability of existence of k linearly independent vectors in a set of vectors with i.u.d. Bernoulli 
entries, we are able to derive a lower bound on the probability that the decoding is successful at the sink 
node in each case. 

Analysis [Details]: Every node combines its received packets in a random fashion. Intuitively, the 
likelihood of linear dependence among the global encoding vectors of the packets increases as they travel 
down the links from node s to node t. 
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Fig. 5. Two examples of a network of length 2 with all-in-all-out schedules of capacity 4 (with and without re-ordering of received packets). 

In the following, we derive a lower bound on the number of received packets | by any receiving 
node V among which with a given probability, k packets are innovative. (There exists an innovative 
collection Nv at node v such that \Nv\ = k). 

Over an edge e = (u,-, Vt-i), a packet is a function of {yUej '■ j ^ -Ms} and for all j G Aig, l^e,j is 
itself a function of {\e,ilJ^i,j ■ i G X^^}. For all i G X^^, Ae,i's are i.u.d., and for all j G Aig, l^i/^ are 
known at node n, and no longer random. Thus, for all j G M.s, fJ'e/^, and thus the entries of Q^, are 
uniformly but not necessarily independently distributed. Let r(Q^) represent the rank of the matrix 
over F2. Clearly, r(Q^) is a random variable. We are interested in finding lower bounds on the probability 
that r{Qy) = k, i.e., the probability of receiving an innovative collection of size k of packets by any 
receiving node v, when n packets are received at the node. While finding such lower bounds is rather 
simple where Q^'s entries are all i.u.d., the same cannot be said about the cases where Qi.'s entries are not 
necessarily i.u.d.. Our approach is thus to derive the bounds by just focusing on a subset of the packets 
in each node for which the entries of the global encoding vectors are i.u.d. To perform this, we remove 
a minimal collection of rows in Q^, so that the remaining rows all have i.u.d. entries. We denote the set 
of remaining rows by Q'^; a sub-matrix of Q^,. Clearly, r{Q[,) < r{Qi,). 

Let Q be an n X A; matrix over F2. A maximal collection of rows in Q with i.u.d. entries is called 
dense, and Q is called a dense matrix if all its rows form a dense collection. We refer to the number of 
rows in a dense collection of rows in Q as the density of Q, denoted by V{Q), and refer to each row in 
this collection as a dense row. 

Let be a decoding matrix of size x /c (at a receiving node v). The set of in-edges of node v 
pertaining to the Q^'s dense rows is denoted by "Dy; and Q.„ restricted to its V{Qu) dense rows is denoted 
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by Q'^ of size 'D{Q,u) x k. The packets whose global encoding vectors are in the set of dense rows are 
called dense packets. 

It should be clear that any packet sent by node s is dense, i.e., V(QyJ = n. The density of the 
decoding matrices at the other nodes further down in the sequence of network nodes (including the sink) 
may however be less than n. 

The collection of rows of which are not in the collection of dense rows is denoted by a sub-matrix 
of Qy \ i.e., the rows of are {/x^ : e G X^,^ \ The minimal collection of rows of Q„ \ V-u whose 
removal would create a sub-matrix with linearly independent rows is denoted by a sub-matrix V[, of Q^; 
i.e., the rows of "P,', are {/Xg : e G 'D^,\N'.u}, where A/^, is a maximal innovative collection of packets in V.^. 
When it causes no confusion, we adopt the same notation V^, (or V^) for the set of in-edges pertaining 
to the rows in Vv (or V[). 

We denote the maximal innovative collection of packets in X^,^ by X„. The number |X„^ \X^;| of non- 
innovative packets at every receiving node v can be bounded above by the sum of the number of rows in 
Vv and V'y, i.e., |Xu^ \ Vy\ + \Vy \ A/'t,|. The number of rows in Vv and V'^ are random variables and we 
shall give upper-bounds on |Xy^ \ and \Vi, \ A/l,| which hold with a given probability. 

For two adjacent nodes u and v connected by the link (m, v) in the network, we have Qi. = T^Qu, 
where is an n x n matrix over F2, whose rows are local encoding vectors {Ag : e G X„^}. For all 
k G X„^, {Tu)e,k = ^e,k- We call the transfer matrix at node u. 

The following lemmas provide a lower bound on V{Q^) = V{TuQu)- The proofs are given in Ap- 
pendix m 

Lemma 1: Let w be a column- vector of length d whose entries are independently and uniformly drawn 
from F2, and T be any hx d (h < d) matrix over F2 so that r(T) = h. Then, the entries of Tu are i.u.d. 
random variables over F2. 

It follows from Lemma [H that a set of linear combinations of i.u.d. random variables over F2 are i.u.d., 
so long as the coefficient vectors of the linear combinations are linearly independent. 

Lemma 2: Let M be a (i x (A; < c/) dense matrix whose entries are drawn from F2, and let T be a 
matrix over F2 with d columns. Suppose that r(T) > h. Then V(TM) > h. 

Lemma[2]is applicable if the decoding matrix Q„, at the transmitting node u, is dense, i.e., X'(Q„) = n. 
The density of Q„ might however be less than n, i.e., Qu might not be dense. 

Remark 1: Lemma [2] implies that if at a transmitting node u, the transfer matrix Tu is not full row-rank. 
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at the receiving node v, there are rows in the decoding matrix Q.^, which are not dense. This means that 
a packet sent by node u would not be in the collection of dense packets at node v if its local encoding 
vector, and thus its global encoding vector, is linearly dependent to those of dense packets at node v. 

Suppose that V{Qu) = d, or equivalently, \Vu\ = d. We rewrite T^Qu with respect to Q'^ being Qu 
restricted to its dense rows. This results in = T^Q'^, where Q'^ is a dense matrix of size d x k, and 
is a matrix of size nx d, so that for all e G and all k e V^, {Tl^)e,k = K,k + Z]igx„^\D„ K,iK,k- 
We call the modified transfer matrix at node u. 

Every row of T„ is the local encoding vector of a packet transmitted by node n, and every entry of 
a local encoding vector is either zero or chosen independently and uniformly at random from F2. Thus, 
the entries of T„ are each either zero or an i.u.d. random variable. For each i E Ou^, and each j E X„^, 
let the sets Til'^^ and Tu^, denote the label sets of i.u.d. entries in the ith row and the jth column of Tu, 
respectively. 

For worst-case schedules, there are at least i packets received by an interior node u by the time that the 
ith coded packet is to be transmitted. Thus, for all i E Ou^, |7^ti| > i, and in particular the first i entries 
of the ith row of T„ are i.u.d. random variables. It can also be seen that for all j E X„^, \Tu;!j\ > n—j + 1, 
and in particular the last n — j + 1 entries of the jth column of Ta are i.u.d. random variables. 

We now consider the modified transfer matrix T^. The ith row of is representing the labels of dense 
packets received by node u which are contributing to generate the ith coded packet to be transmitted. The 
jth column of is also representing the labels of coded packets in which the jth dense packet at node 
u is contributing. For all e E Ou^, and all k E V^, {Tli)e,k is either zero or an i.u.d. random variable. 

For each i E O,,, , and each 7 E V,,, we use the notations T'i*'' and T'^,P to denote the label sets of 
i.u.d. entries in the ith row and the jth column of T^, respectively. 

One can see that for all i E Ou^, I'T'uij — [i — n + d]'^, and in particular the first [i — n + d]'^ entries of 
the ith row of are i.u.d^ Also, for all j E X^^ , {T'^l, \ > d — j + 1, and in particular the last d — j + 1 
entries of the jth column of T'^ are i.u.d.. 

In what follows, we derive lower bounds on the rank of T^. It is worth noting that the rank property of 
such a matrix is highly dependent on the schedule. Thus the analysis of one-in-one-out and all-in-all-out 
worst-case schedules are given separately in the following. 

^For every integer x, [x]'^ = a;, if a; > 0, and [x]'^ — 0, otherwise. 



16 



One-In-One-Out Worst-Case Schedules: In this setting, for all i G \Tul}^\ = i, and for all j G . 

|r£!| = n-j + 1. Also, for all i e O^^, {T'^J > {i-n + d)+, and for all j G X„^, IT'gj > rf- j + it 
Lemma 3: Let T be an n x (d < ra) matrix over F2, so that for any 1 < j < rf, at least d — j + 1 
entries of its jth column are i.u.d. random variables. The rest of the entries are set to zero. For any integer 
< 7 < - 1, then 

Pr[r(T) <d--f]<{d- 7)2-(^+^). 
The proof of Lemma |3] is given in Appendix HI 

Lemma [3] is applicable to T^, and since Qi, = T'^Q'^, and Q'^ is dense. Lemma ID gives V{Qy) > h, for 
every h such that r(T^) > h. Combining Lemmas [2] and |3l we can hence give a lower bound on V(Qi,). 

Lemma 4: For any e > 0, applying a dense coding scheme over a network with any one-in-one-out 
worst-case schedule of capacity n, for any network link {u,v), the inequality 

V{Q,) > V{Q^) - \ogV{Qu) - log(l/e) 

fails with probability (w.p.) bounded above by (b.a.b.) e. 
The proof of Lemma |4] is given in Appendix HI 

For any network link {u, v), V{Qu) — ^{Qy) is called density loss at node v, or over the link (n, v). 
Lemma m gives an upper bound on the density loss at node v with respect to V{Qu), i.e., w.p. bounded 
below by (b.b.b.) 1 - e, V{Q^) - V{Q„) < \ogV{Qu) + log(l/e). 

The density of the decoding matrix at node t, V(Qt), can be bounded from below by subtracting the 
density losses over the network links from the density of the decoding matrix at the first receiving node. 
The proof of the following is given in Appendix HI 

Lemma 5: For any e > 0, applying a dense code over a network of length / with any one-in-one-out 
worst-case schedule of capacity n, 

V{Qt) >n-l\og{nl/e) 

w.p. b.b.b. 1 — e. 

Lemma [5] provides an upper bound on the number of rows in Vt, as \Xt^ \ X'^l = n — V{Qt). 
Now, let Q[ be a sub-matrix of Qt, so that Q[ includes all V{Qt) dense rows in Qt. By Lemma [51 
the probability of V{Qt) < k h upper bounded hy e, ii k < n — l\og{nl/e). Thus, for every k satisfying 

'The results for the matrices T and T' are consistent, in that the results for T are the special case of the results for T' , where d = n. 
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this inequality, w.p. no larger than e, Q[ fails to be a V{Qt) x k (k < V{Qt)) dense matrix. Finally, the 
following lemma gives an upper bound on the probability that a dense matrix fails to have rank k (see 
Appendix H] for the proof). 

Lemma 6: Let M he a d x k (k < d) dense matrix over F2. Then, for every e > 0, 

Pr [r(M) <k]<e, 

so long as k < d — log(l/e). 

Lemma |5] together with Lemma |6] give an upper bound on the total number of packets at node t not 
belonging to a maximal innovative collection of packets, i.e., \It^ \ It\, by respectively providing upper 
bounds on the number of rows in Vt and V^, i.e., \Xt^ \ Vt\ and \Vt \ A/t|. This is used in the proof of 
the following theorem. 

Theorem 1: For any e > 0, a dense code over a vector space , for any integer L, fails to be capacity- 
achieving for a network of length / with k message packets under any one-in-one-out schedule of capacity 
n, w.p. no larger than e, so long as 

k<n -l\og{nl/e) -log(l/e) - / - 1, 

and /log(//e) = o{n). 

Proof: The decoding fails if the number of innovative packets at node t is less than k. Let e denote 
e/2, for the simplicity of exposition. Replacing e with e in Lemma [51 the number of dense packets at node 
t is less than n — l\og{nl/e) w.p. b.a.b. e. Given that at least n — l\og{nl/e) dense packets are received 
by node t, based on Lemma [6l there is less than k innovative packets among the set of dense packets 
w.p. b.a.b. e, so long as k < n — l log{nl/e) — log(l/e) — I — 1. Hence, the probability of decoding failure 
is b.a.b. e. Thus, for this scenario, a dense code is capacity-achieving, i.e., the code rate k/n goes to 1, 
as n goes to infinity, so long as /log(//e) = o(n). ■ 
All-In-All-Out Worst-Case Schedules: In all-in-all-out worst-case schedules, for all i G Cu^, |7^ti| = n, 
and for all j G X,^, |r£!| = n. Similarly, for all ^ G a^, \T't]j = d, and for all j G X„^, IT'i^ij = n. 
Since these conditions on |T'i*2wl ^^'^ l^'ilil satisfy those required in Lemma |3l one can use the result of 
Lemma [3] to upper bound Pr [r(T') < d — 'j], for any integer < 7 < d — 1. We, however, derive a tighter 
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bound for this setting in Lemma ItI^ The proof of the following lemmas can be found in Appendix IB 
Lemma 7: Let T be a n x (i < n) dense matrix over F2. For any integer 1 < 7 < (i — 1, 

Pr[r(r) < d-7] < 2"^. 

We, now, give a lower bound on V{Qy) = V{Tl^Q'^) by using Lemmas |2] and U\ 
Lemma 8: For any e > 0, applying a dense coding scheme over a network with any all-in-all-out 
worst-case schedule of capacity n, for any network link {u,v), the inequality 

p(gj >i?(g„)-iog(i/e) 

fails to hold w.p. b.a.b. e. 

Taking a union bound over the number of links in the network, and subtracting the density losses over 
the network links from the density of the decoding matrix at the first receiving node, a lower bound can 
be given on the density of the decoding matrix at node t, i.e., V{Qt). The proof of the following is similar 
to that of Lemma |5l except that we use the result of Lemma [8] instead of Lemma IH 

Lemma 9: For any e > 0, applying a dense code over a network of length / with any all-in-all-out 
worst-case schedule of capacity n, 

ViQt) >n-l\og{l/e) 

w.p. b.b.b. 1 — e. 

Applying Lemmas |6] and |9] yields the following main result (The proof is similar to that of Theorem [T] 
and is thus omitted). 

Theorem 2: For any e > 0, a dense code over a vector space F2 , for any integer L, fails to be capacity- 
achieving for a network of length / with k message packets under any all-in-all-out worst-case schedule 
of capacity n, w.p. no larger than e, so long as 



k<n-l log(//e) - log(l/e) - / - 1, 



and / log(//e) = o{n). 



'"in the case of 7 = 0, Pr[r(r) < d — 7] can also be bounded by Lemma[6l in this case, the result of Lemma|7]is not as tight as that of 
Lemma [6] However, Lemma |6] cannot be generalized to the cases ofO<7<ci— 1, and hence Lemma [7] can be considered as a complement 
to Lemma [6] 
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Coding Costs: The worst case with regards to the encoding cost at any interior node u (node s) occurs 
if for any time r at which node u transmits a packet, |X„^ | = n, and for all i E X^^, Ag j's (for all j E Aig, 
/iej's) are chosen to be nonzero. Thus the number of packet operations for encoding at any interior node 
(the source node) is O(n^) (0{kn)). Thus the encoding cost is 0{n)\^ 

To solve the system of linear equations at node t using Gauss-Jordan elimination, node t requires 
0{wn) row operations, where w is the widest bandwidth of a row vector of Qt- Note that a row operation 
is equivalent to at most 0{k) field operations along with at most one packet operationo Thus node t 
requires 0{wn) packet operations together with Oiwkn) field operations flU. 

The worst case regarding the decoding cost at node t occurs if w equals k. Thus the number of 
operations for decoding is 0{k'^n) field operations and 0{kn) packet operations. Thus the decoding cost 
is also 0{n). 

Theorem 3: The encoding and decoding costs of a dense coding scheme over a line network with any 
worst-case schedule of capacity n are 0{n). 

We have shown that the class of dense codes is capacity-achieving over networks with arbitrary schedules 
(Theorems \T\ and |2l) but at the cost of large computational complexity (Theorem |3]). 

To reduce the coding costs, the method of generating coded packets has to be modified so that the 
global encoding vectors have smaller bandwidth. The smaller is the bandwidth of the global encoding 
vector of a packet, however, the smaller is its randomness and the larger is its probability to be linearly 
dependent on the global encoding vectors of the other packets sharing the same band. Thus the probability 
of a packet being innovative may become smaller in general. The problem is therefore to design coding 
schemes in which every global encoding vector has a small bandwidth but the bands are set up in a way 
to compensate for the reduction in the randomness of the global encoding vectors. 

To have global encoding vectors with smaller bandwidth, the general approach is to apply a dense code 
to a chunk, a smaller sub-message of the original message. In fact, for the so-called chunked codes (CC) 
[[HI, the set of message packets is partitioned into chunks of equal size, and each chunk is transmitted 
by a dense code. In a general context, the design of a chunk-based code has to deal with the following 
issues: (i) how to divide the message packets into the chunks at the source node, (ii) how to schedule 
the chunks to be coded and transmitted by the network nodes, and (iii) how to recover each chunk at the 

"The number of packet operations for encoding is at most (Z — l)ri^+nfc, and hence the encoding cost is at most {l — l)n^/kl+n/l — Oin), 
since n/k — 0{1), as n goes to infinity. 

'^The narrowest window (in an end-around fashion) within which the non-zero entries of a vector he is called band. The length of a band 
is called bandwidth. 
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sink node. 

In the following, we review the CC scheme of (HI, which provably performs well over the networks 
with arbitrary schedules. 

B. Chunked Codes 

Suppose that node s is given a set of message packets {xi : 1 < i < k}, each of which is a vector 
in vector space Let A^s be the set of labels of packets in Ai. 

A CC operates by dividing the set Ai into q disjoint subsets {Aiui : to E [q]}, called chunks, each of 
size k/q (i.e., \Aiui\ = k/q), so that Aiu) includes the last k/q contiguous message packets whose labels 
are equal to or less than cuk/q. The size of chunks is referred to as the aperture size. 

Encoding: Every transmitting node u at any time r, randomly chooses a chunk, say uj, and constructs 
a coded packet, called an cu-packet, by randomly linearly combining all the w-packets received earlier by 
node u and transmits it over an out-edge of node Ur, i.e., each chunk is transmitted by using a dense 
code. 

Let xt!-^ (Oit!^ be the set of (labels of) in-edges (out-edges) of node u prior to time r over which 
cu-packets are received (sent), and let A4i'^^ be the set of labels of message packets in chunk 00. 

Therefore, for every out-edge e of node Ur, a chunk is randomly chosen, say cu, and an cj-packet y^, is 
sent so that = Xl^gif") ^e,iyi if u is interior, and y^ = ^j^j^ij^) fJ-ejXj if u is the source, where for all 
i G 1^2 \ ^e,i is 0; and for all i G and all j G Mf'\ Xe,i and fiej are symbols independently 
and uniformly drawn from F2. 

Let J\fv be a subset of X^,^. Suppose a collection {y^ : e G Mv} of packets at a receiving node v. We 
refer to such a collection of packets by its set of in-edges A/^,. Let A/'i'^^ be the set of all w-packets in A't,. 
We say that a collection ATi") (or a;) is tu-innovative (or innovative) if the global encoding vectors of 
the packets therein are linearly independent. Since for all j G Al^ \ Ali'^\ the jth entry of an w-packet's 
global encoding vector is 0, A/'„ is innovative iff A/'i'^'' is cu-innovative, for every oj such that J\fy contains 
at least one w-packet. 

Decoding: For every u, node t has to solve a system of linear equations {y^ = '^j^j^(^) l^e,j^j '■ e G 
for k/q packets {xj : j G Ali'^^}. The system is uniquely solvable for every uj, and for all j G M.^^\ 
Xj equals Xj if there exists an w-innovative collection A/'/'^'' at node t such that |A/'/'^''| = k/q. Thus, a 
CC succeeds if for every cu, node t receives a collection of k/q cj-packets which form an cj-innovative 
collection. 
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Fig. 6. An example of a network of length 2 with a schedule of (linear coding) capacity 6. Consider a CC with two chunks cji, and cj2. 
The solid (dashed) lines are the edges over which cji -packets (aj2-packets) are sent. The capacity of the flow by wi-packets is 3, and that 
by (^2 -packets is 2. 

Let be the decoding matrix at a receiving node v. For every u, let called the u-decoding 

matrix at node v, be a |xi^^| x lA^i"^^] sub-matrix of whose rows are the global encoding vectors of 
cj-packets at node v, and whose columns are labeled with M.^^\ 

There exists an innovative collection of k packets at a receiving node v, iff, for every there exists a 
collection of k/q linearly independent rows in 

Suppose that for every 00, an cj-innovative collection Af}'^^ of k/q w-packets is received by node t (i.e., 
r{Qf'^) = k/q). For every lo, node t applies Gauss-Jordan elimination to the extended matrix 
and after removing all-zero rows, it returns a new matrix [I\Z^], where Y^^ is the l^t'^l x 1 vector of 
w-packets at node t, I is a k/q x k/q identity matrix and Z^^ is a k/q x 1 vector each of whose entries 
is a vector in Fg. For all j G M.^s^\ xj equals the jth entry of vector Z^^. 

Analysis: We need to derive a lower bound on the probability of receiving an w-innovative collection 
of k/q cij-packets by the sink node t, for every cu. 

Since for every co, chunk cu is of size k/q and is transmitted by a dense code, we can use the result of 
Theorems [Hand I2] by replacing k and n, respectively, with k/q and the capacity of the schedule pertaining 
to the w-packets. We refer to such a schedule as u-schedule. The capacity of the cj-schedule is a random 
variable. This capacity is equal to the number of paths between sq and too whose (traffic) edges, which 
carry cj-packets, are disjoint. Such paths are referred to as flow paths (see Figure |6l). 

To deal with the randomness of w-schedules, we derive an upper bound on the number of w-packets 
received by any interior node that cannot be "matched up" with an w-packet which is transmitted 
subsequently by the node and is not yet coupled with an w-packet received earlier by the node. Those 
w-packets received by the node whose (traffic) edges do not contribute in a maximal collection of flow 
paths are called unusable, and the rest of w-packets are called usable. The term "unusable" reflects the 
fact that these packets are not part of the flow paths that contribute to the capacity of the w-schedule. 
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One-In-One-Out Worst-Case Schedules: For this scenario, we take a union bound over interior nodes 
by subtracting the number of unusable w-packets received by every node from the number of usable 
cj-packets sent by node s. This allows us to derive a lower bound on the number of usable cj-packets at 
node t yielding a lower bound on the capacity of the w-scheduleQ 

Lemma 10: For a CC with q chunks, over a network of length / with any one-in-one-out worst-case 
schedule of capacity n, for every to, the capacity of the u;-schedule is less than 

ip={l-0(^{l'iq/n)\n{ln/e)f')y{n/q) (1) 

w.p. b.a.b. e, so long as 

Iti 

Pq In— = o{n). (2) 

Proof: Please refer to Appendix UIl ■ 
We, now, derive a lower bound on the capacity of schedule, n, so that with a given probability, for every 
CO, there is an innovative collection of k/q w-packets at node t. For this, we use Theorem [T] To modify 
the result of Theorem [Uto be applicable to the transmission of a given chunk oj over a line network by 
a dense code, we replace k and n with k/q and respectively. Then the following is immediate. 

Lemma 11: For any e > 0, in a CC with q chunks, a given chunk u (of size k/q) fails to be successfully 
sent through a line network of length / with any one-in-one-out worst-case schedule of capacity n, by 
using a dense code, w.p. no larger than e/g, so long as 

k/q < f — I \og{nl/e) — log(g/e) — / — 1, 

and Pq\n{nl/e) = o{n), where (/? is the capacity of the cj-schedule. 

The decoding at node t is successful if and only if every chunk can be decoded. Taking a union bound 
over the number of chunks (g), we upper bound the number of message packets (k) at node s, such that 
with a given probability, for every chunk to, there exist at least n/q — {{l^{n/qY ln(/n/e))^/^) w-packets 
at node t, such that k/q w-packets form an cj-innovative collection. This yields the following for CC over 
networks with one-in-one-out worst-case schedules. 

Theorem 4: For any e > 0, a CC with q chunks, over any vector space , is capacity-achieving for a 
network of length / with k message packets under any one-in-one-out worst-case schedule of capacity n, 

'^One should note that Lemma [Tol is different from fS] Theorem 4.1] in the sense that the lower bound derived here is tighter, though the 
proofs have generally a similar structure. 
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with failure probability no larger than e, so long as 

k < qip — ql\og{nl/i) — glog(g/e) — ql — q, 
and /^gln(n//e) = o{n), where 

ip = n/q-0 {{l%n/qy\n{ln/e)y/^) . 

Proof: Replacing e with e = e/2 in Lemmas [TOl and [TTl we obtain (i) for a chunk cu, the capacity of 
the cj-schedule fails to be at least n/q — ((/'^(n/g)^ ln(/n/e))^/'^), w.p. b.a.b. e, so long as g/'^ln(n//e) = 
o{n); and (ii) given that every chunk cu has been allocated a flow with capacity (p of at least n/q — 
O {{p'{n/qY\\i{ln/ e)Y/^^, a dense code fails to transmit the chunk u successfully over the network 
w.p. b.a.b. e, so long as k < (pq — ql \og{nl/e) — q log(g/e) —ql — q. Thus, the probability of failure, when 
either (i), or (ii) occurs, is b.a.b. e. ■ 

All-In-All-Out Worst-Case Schedules: By the nature of the all-in-all-out schedule, it is easy to see that 
the capacity of an w-schedule is equal to the minimum number of w-packets transmitted over all the links 
in the network. Using this, we obtain the following lower bound on the capacity of an u;- schedule. 

Lemma 12: For a CC with q chunks, over a network of length / with any all-in-all-out worst-case 
schedule of capacity n, for every to, w-packets fail to form a flow of capacity </? larger than 

(l-0(((g/n)ln(/g/6))^/')).(n/g) (3) 

w.p. b.a.b. e, so long as 

Iq 

g In — = o{n) . (4) 
e 

Proof: Please refer to Appendix [nil ■ 
The proofs of the following results are similar to that of Lemma \TT\ and Theorem HI except that in this 

case, we use Lemma [T2] and Theorem |2] instead of Lemma [JO] and Theorem [T] respectively. 

Lemma 13: For any e > 0, in a CC with q chunks, a given chunk to (of size k/q) fails to be successfully 

transmitted through a line network of length / with any all-in-all-out worst-case schedule of capacity n, 

given that the oj-packets form a flow of capacity larger than n/q — O [{{n/q) ln(/g/e))^/^), by using a 

dense code, w.p. no larger than e/q, so long as 



k/q < ip -nog{lq/e) - \og{q/e) - l-l, 
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and gln(/g/e) = o(n). 

Theorem 5: For any e > 0, a CC with q chunks, over any vector space , is capacity-achieving for 
a network of length / with k message packets under any all-in-all-out worst-case schedule of capacity n, 
with failure probability no larger than e, so long as 

k < qip — ql\og{lq/e) — glog(g/e) — ql ~ q, 

and lqhi{lq/e) = o{n), where 

^ = n/q-0{{{n/q)\n{lq/e)y/'). 

Coding Costs: The worst-case with regards to the encoding cost occurs if for every cu, any transmitting 
interior node u (or the source node s) sends all its successful cj-packets after receiving all w-packets sent 
by the node in the upper layer, and if all the local encoding coefficients are chosen to be non-zero. In 
this case, the number of packet operations for encoding at any interior node u (or node s) is 0{n^ /q) (or 
0{kn/q)). (There are 0{n/q) oj-packets sent and received by any interior node u and there are 0{n/q) 
cj-packets sent by node s.) Thus the encoding cost is 0(n/g)Q 

To solve the system of linear equations {y^ = fJ'ejXj : e G xf^^} for k/q packets {xj : j G 

using Gauss-Jordan elimination, node t requires 0{kn/q) row operations for every u (the band- 
width of a row in an cj-decoding matrix is at most k/q). Thus, node t requires 0{k'^n/q'^) field operations 
along with 0{kn/q^) packet operations (the Qt'^'^'s bandwidth and size are k/q and 0{n/q) x k/q, 
respectively). The total number of operations for decoding at node t is 0{k'^n/q^) field operations and 
0{kn/q) packet operations, and thus the decoding cost is 0{n/q). 

Theorem 6: The encoding and decoding costs of a CC with q chunks are 0{n/q). 

C. Comparison of Dense Codes and Chunked Codes: Making the Case for Overlapping Chunks 

By comparing the results of Theorems [T] and |2] with Theorems |4] and [5l respectively, one can observe 
that, for each type of worst-case schedules, the overhead per message, (n — k)/k, for CC is larger by a 
factor of q compared to dense codes. This is the price that one has to pay for the lower coding complexity 
of CC. The question now is whether the tradeoff between the speed of convergence and coding complexity 
of CC can be improved. 

'■'The number of packet operations for encoding is 0{{l—l)n^ /q)+0{kn/q), and hence the encoding cost is 0{{l—l)n^ /qkl)+0{n/ql) — 
0{n/q), since n/k = 0(1), as n goes to infinity. 
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The analysis of CC over networks with arbitrary schedules shows that the performance of CC is affected 
by (i) the capacity of the cj-schedule for every cu; (ii) the number of innovative packets in the collection 
of dense packets, and (iii) the condition of decoding completion, i.e., for all cu, the cj-decoding matrix 
Qt'^'' at node t needs to have k/q linearly independent rows (i.e., r{Q['^^) = k/q). 

In the scenario of interest in this work, where the network nodes are blind to the schedule, and there is 
no feedback information available to the nodes, random coding appears to be the best strategy as far as 
issues (i) and (ii) are concerned. There is however room for improvement in the performance of CC by 
modifying the chunking scheme to speed up the decoding process. The main idea is to devise a chunking 
scheme such that for every 00, r{Qf''^) does not necessarily have to be k/q for r{Qt) = k. We demonstrate 
that this can be achieved by allowing chunks to overlap. 

To explain the idea, we start by the simple case of a single "erasure channel" with an arbitrary schedule. 
It is important to note that in this case, the lack of interior nodes significantly simplifies the analysis as 
one does not need to consider density losses over the network links, i.e., all the received packets at node 
t are denseO The following results are simple to prove. 

Theorem 7: For any e > 0, a dense code over a vector space F2 , is capacity-achieving over an erasure 
channel with k message packets under any schedule of capacity n, with failure probability no larger than 
e, so long as < n — log(l/e) and log(l/e) = o{k). The encoding and decoding costs are 0{n). 

Theorem 8: For any e > 0, a CC with q chunks, over any vector space , is capacity-achieving over an 
erasure channel with k message packets under any schedule of capacity n, with failure probability no larger 
than e, so long as k < q(p — q\og{q/e) — q, and glog(g/e) = o(n), where cp = n/q — {{{n/q) ln(g/e))-'^/^). 
The encoding and decoding costs are 0{n/q). 

The comparison of the results of the two theorems shows that for a single erasure channel also, CC 
have a slower convergence to capacity than dense codes. This is the cost for having a lower complexity 
by a factor of q. 

Recently, Studholme and Blake ^ introduced a class of erasure codes called windowed erasure codes, 
which is similar to CC except that for windowed erasure codes, the chunks are allowed to overlap. The 
codes are used in [|9l to deliver k message packets over an erasure channel with any schedule of capacity 
n. To perform this task, windowed erasure codes operate on k chunks of aperture size a, where any two 
contiguous chunks overlap in all but one message packet in an end-around fashion. Similar to CC, in 
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It is worth noting thiat for a single erasure cliannel witli an arbitrary sciiedule, a CC performs better than what is presented in (8), in 
terms of the speed of convergence, for a given coding cost. 
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windowed erasure codes, the chunks are scheduled at random, i.e., the source node at any time instant 
randomly chooses a chunk, constructs and transmits a coded packet by randomly linearly combining the 
message packets in the chosen chunk. The decoding of windowed erasure codes is, however, similar to 
that of dense codes, not that of CC, i.e., the sink node uses Gauss-Jordan elimination to solve the system 
of linear equations for all the chunks together. 

The following theorem shows that windowed erasure codes can achieve the capacity of erasure channels 
with arbitrary schedules at the same speed as dense codes. 

Theorem 9: For any e > 0, a windowed erasure code with any aperture size a > 2\/k, over a vector 
space , is capacity-achieving over an erasure channel with k message packets under any schedule of 
capacity n, with failure probability no larger than e, so long as k < n — log (1/e), and log(l/e) = o{k). 
The encoding and decoding costs are 0{a). 

Prior to proving Theorem |9l we introduce two categories of structured random matrices. 

Banded Random Binary Matrices: Let n, k, a and 7 be integers (/c < n, 7 < a), so that a — 7 is a 
divisor of k. Let x be A;/(a — 7), and / be the set of integers in [k]. We divide / into x subsets //s, for 
all i E [x], where Jj (the ith aperture of size a) is the set of a contiguous integers in / in an end-around 
fashion, starting from (? — l)(a; — 7) + L 

We construct an n x /c matrix as follows: (i) for each row, an index, say i, is randomly chosen from the 
set of integers [x], and (ii) the row's entries indexed by the ith aperture are independently and uniformly 
chosen from F2, and the rest of the entries are set to zero. We call such a matrix a (7, a) irregular 
symmetric banded matrix of size n x k. Now, consider a similar construction, with the difference that 
a — 7 is now a divisor of A; — 7 (not k), and x is (k — 'j)/(a — 7) (not k/{a — 7)). The resulting matrix 
in this case is called a (7, a) irregular asymmetric banded matrix of size n x k. Also, consider a matrix 
constructed as each of the above two procedures, except that in part (i), each index in [x] is assigned to 
n/x rows (x h^s to be a divisor of n). We call such a matrix a (7, a) "regular" symmetric or asymmetric 
banded matrix of size n x k. 

Proof of Theorem |9l' Each packet received by node t pertains to a randomly chosen chunk. Each 
chunk contains a set of a contiguous indices in an end-around fashion from the set of integers [k]. Each 
packet's global encoding vector has random entries in the positions indexed by the aperture pertaining 
to the chosen chunk, and the rest of the entries are zero. Thus, the decoding matrix Qt at node t is an 
(a — 1, a) symmetric banded random binary matrix of size n x k. 
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The decoding at node t will be successful if Qt has rank k. The following result which is a short 
version of Conjecture 4.2] is useful to upper bound the probability that Qt fails to have rank k, i.e., 

PT[r{Qt) < k]. 

Conjecture 1: Let M be an {a — l,a) (regular/irregular) symmetric banded random binary matrix of 
size n X k (k < n). For any e > 0, and for sufficiently large k, Pr [r(M) < k] < e, so long as 

k <n — log(l/e), 

and a > 2\/k. 

By Conjecture[il the probability of failure of a windowed erasure code is b.a.b. e, i.e., Pr[r((5t) < k] < e, 
so long as k < n — log(l/e). By definition, a coding scheme is capacity-achieving if the ratio knmax/n 
goes to 1, as n goes to infinity. Here, A;„max = n — log(l/e), and kn^^^/n goes to 1, as n goes to infinity, 
so long as log(l/e) = o{n). This completes the proof of the first part of the theorem. We prove the second 
part regarding the coding costs in two steps. The proof that the encoding cost is 0{a) is similar to that 
of Theorem |6l To prove the decoding cost, it suffices to recall that applying Gauss-Jordan elimination to 
the matrix Qt of bandwidth of a, node t requires at most 0{an) row (or packet) operations. Thus the 
decoding cost is 0(a), since n/k = 0(1), as n goes to infinity. ■ 

The comparison of the results of Theorem 9 with those of Theorem 7 indicates that for the transmission 
over a single erasure channel, windowed erasure codes with sufficiently large apertures achieve the 
capacity at the same speed as dense codes do but with lower complexity. This motivates the application of 
chunked codes with overlapping chunks, referred to as overlapped chunked codes (OCC), to the problem 
of information transmission over erasure networks with arbitrary schedules. 

D. Overlapped Chunked Codes 

Suppose that node s is given a set of A; message packets {xi : 1 < z < k}, each of which is a vector 
in F2 , for any integer L. 

An OCC operates by dividing the set M. into q overlapping chunks {M.uj '■ ^ ^ [o]}^ each of size 
a (i.e., \M.uj\ = Ci), so that any two contiguous chunks overlap by 7 = a — k/q message packets in 
an end-around fashion. The set of labels of the message packets in a given chunk uj, A^^ is called the 
aperture of chunk uj. 

To ensure that all the message packets appear in the same number of chunks, (a — 7) must be a divisor 
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of a. For the simplicity of exposition, we consider 7 = a/rg, where Tg = t/{t — 1), for any divisor r of 
tt(l<T<a)o For instance, there is no overlap between chunks when r is equal to 1; and the overlap 
becomes larger as r increases; namely, when r is equal to a, any two contiguous chunks overlap in all 
but one message packet. We call r the overlap parameter. 

Encoding: The encoding is performed similar to CC; i.e., every transmitting node u at any time instant 
r' chooses a chunk at random, say w; and constructs an w-packet, by randomly linearly combining the 
w-packets already received by node u and transmits it over an out-edge of node n^/. 

It is important to note that for OCC, unlike CC, the w-packets for different values of u with overlapping 
apertures, are not necessarily linearly independent. 

Decoding: The decoding is performed similar to dense codes; i.e., node t has to solve a system of 
linear equations {y^ = J2j(^Ms • *^ ^ ^too} for ^ packets {xj : j G Ms}- The system is uniquely 

solvable, and for all j E Ais, Xj is equal to xj, if there exists an innovative collection of k packets at 
node tllZ 

Suppose that a collection of k innovative packets are received by node t, i.e., r{Qt) = k. To solve the 
system of linear equations node t applies Gaussian elimination to the extended matrix and after 

removing all-zero rows it returns a new matrix [I\Z], where Y is the n x 1 vector of the packets at node 
t, I is a k X k identity matrix and Z is a A; x 1 vector each of whose entries is a vector in . For all 
j E Ms, Xj is equal to the jth entry of vector Z. 

Analysis: Similar to CC, we analyze OCC over the two extremal types of worst-case schedules 
separately. 

One-In-One-Out Worst-Case Schedules: The chunks are scheduled in OCC similar to CC. Thus, for 
every u, w.p. b.a.b. e, the capacity of the flow by w-packets (w-schedule) fails to be larger than the lower 
bound given in Lemma \T0\ However, for successful decoding, unlike CC, in OCC, a given chunk cu does 
not have to be necessarily recoverable in isolation (i.e., r{Qf^^) does not need to be a). This is because 
the decoding is performed on the set of all the packets received by node t. The goal is to derive an upper 
bound on k, such that r(Qt) equals k, w.h.p., so long as n packets are received by node t. 

We first lower bound the total number of dense w-packets, given that n packets are received by node 

''The parameter r, defined as the overlap parameter here, should not be mistaken with the same notation for the transmission time instances 
used earlier. 

"it is worth noting that in prior related works, the chunks are to be decoded in isolation. However, by performing the decoding algorithm 
on the set of all the chunks simultaneously, the decoding of the OCC may be successful even when none of the chunks are recoverable in 
isolation. Thus, a smaller number of packets at the sink node is sufficient to ensure successful decoding with a given probability of success. 
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t, SO that, for all cu, the lower bound fails to hold w.p. b.a.b. We then derive an upper bound on k, so 
that, w.h.p., for all cu, there exists an innovative collection of k packets among the union set of maximal 
collections of dense cj-packets. 

Lemma 14: For any e > 0, applying a dense code to a given chunk co over a network of length / with 
any one-in-one-out worst-case schedule of capacity n, given that the capacity of the flow by cu-packets is 

the number of dense w-packets at node t is less than — /log(</?//e) w.p. b.a.b. e. 
Proof: This is a direct result of Lemma |5l by replacing n with y?. ■ 

Lemma 15: Let l,n,q,Lp and e be defined as above. Then the node t fails to receive at least if — 
l\og{q(pl/e) dense cj-packets, for every u, w.p. b.a.b. e, and ip fails to be at least 



l-0((/^(g/n)ln(/n/6))'/')).(n/g), 



w.p. b.a.b. e, so long as 



Iti 

/^gln— = o(n). (5) 



Proof: Replacing e with e/q in Lemma [141 for taking a union bound over q, the first part of the 
lemma follows. Replacing e with e in Lemma [TOl proves the second part of the lemma. ■ 

Thus, so long as Pq\n(ln/e) = o(n), w.p. b.b.b. 1— e, there exist more than n/g—O ((/^(n/g)^ ln(/n/e))^/^) 
w-packets at node t, for every to, and hence there will be more than n/q — O ((/^(n/g)^ ln(/n/e))'^/^) — 
/log(n//e) dense w-packets at node t w.p. b.b.b. 1 — e. 

Let Q'^ be Qt restricted to its rows pertaining to the dense u;-packets, for all to. Thus, is of size 
h X k, for some h smaller than n — O (g(/'^(n/g)^ ln(/n/e))^/'^) — ql\og{nl/ e), w.p. b.a.b. e. Now, the 
problem is to derive an upper bound on the probability that Q[ fails to have rank k. By the structure of 
OCC, it should be clear that Q'f- is a symmetric banded random binary matrix with aperture size a, where 
the overlap size between any two rows pertaining to any two contiguous chunks is 7. Lemma [l5] shows 
that the number of dense a;-packets, for every uo, fails to be larger than h, w.p. b.a.b. e, where 

h = n/q-0 {{l^{n/qf \Ti{ln/e)f'^) - I log(n//e). 

Let Q'l be a sub-matrix of Q[ of size m x k {k < m), so that Q'l includes h {= m/q) rows pertaining 
to each chunk. By the above argument, w.p. b.b.b. 1 — e, such a sub-matrix Q'l of Q[ exists, and Q'l is an 

'^Note that a collection of oj-packets is dense if the entries of their global encoding vectors' entries indexed by the aperture of chunk 
are i.u.d. r.v.'s, and a given cj-packet is dense if it belongs to a maximal dense collection of oj-packets. 
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m X k (j, a) regular symmetric banded matrix. 

We shall derive an upper bound on the probability that fails to have rank k in order to upper bound 
the probability that fails to have rank k. The results of Conjecture [T] cannot be applied to our setting 
in general, since we do not restrict the overlap 7 to be a — 1. Surprisingly, similar result also holds for 
more general settings as it can be seen through our simulation results in Section |V] (no formal proof is 
known yet). We formalize this observation in a conjecture as foUowsQ 

Conjecture 2: Let n, k, a and 7 be integers {k < n, ^ < a). Let M be a (7, a) (irregular/regular) 
symmetric or asymmetric banded random binary matrix of size n x k. For any e > 0, and for sufficiently 
large k, Pr[r(Af) < A;] < e, so long as 

k < n — log(l/e), 

and 7 > 2^/k, or 7 > TeTy/k, respectively, where 7 = a/rg, and Tg = t/{t — 1), for any constant divisor 
r of a. 

Lemma [TSl together with Conjecture |2] (symmetric case) yield the following^ 

Theorem 10: For any e > 0, an OCC with q chunks of size a, and overlap 7 > 2\/k, over any vector 
space F2 , is capacity-achieving for a network of length I with k message packets under any one-in-one-out 
worst-case schedule of capacity n, with failure probability no larger than e, so long as 

k <qh — log(l/e), 

and /^gln(n//e) = o{n), where h = n/q — O {{pin/qY ln(/r;,/e))^/^) — l\og(nl/e) — I. 

Proof: Replacing e with e in Lemma [15] and Conjecture |2l (i) the capacity of the w-schedule, for any 
oj, fails to be at least (1 —0{{l^{q/n) ln(/n/e))^/'^)).(n/g), w.p. b.a.b. e/2, so long as /^gln(n//e) = o(n); 
(ii) given that every cu is allocated a flow of capacity (1 — 0{{P{q/n) \n(ln/e)Y^^)).{n/q), there does not 
exist n/q — 0{{l^{n/qYln{ln/e)Y^^) — l\og{nl/e) dense w-packets at node t, w.p. b.a.b. e/2; and (iii) 
given that node t receives at least the above number of dense w-packets, for every u, the matrix Q[ fails 
to have rank k, w.p. b.a.b. e, so long as k < n — 0{q{P{n/qY lii{ln/e)y^^) — ql\og{nl/e) — ql — log(l/e); 
Thus, the probability of failure, when either (i), or (ii), or (iii) occurs, is b.a.b. e. ■ 
Let T* be the smallest integer divisor of a given aperture size a, such that by choosing r = r*, we get 

"We briefly highlight the differences between Conjecture |2] and Conjecture 4.2 of [gj: (i) the latter considers a sub-class of symmetric 
banded random binary matrices, yet the former considers two more general classes of regular/irregular symmetric and asymmetric banded 
random binaiy matrices, and (ii) unlike the latter, for a given aperture size, the overlap size in the former is not restricted to one particular 
value. 

^"Note that any result based on Conjecture |2] is itself conjectural. 
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7 > 1\fk. (We are assuming that a is chosen in a way that such a r* exists. The case for which there is 
no such r* will be discussed later.) For all r > r*, 7 > 1\fk, and Theorem [TOl holds. The larger is the 
value of r, however, the larger is the number of chunks and the slower would be the speed of convergence 
of OCC to the capacity for a given a (note that fixing a implies fixing the coding costs)c!l Thus, an OCC 
with the overlap parameter r* has the largest speed of convergence to the capacity, for a given aperture 
size, in one-in-one-out worst-case schedules. 

Now suppose that there is no such r* for a given a (i.e., for any integer divisor r of a, 7 < 2v^), 
but there exists at least an integer divisor r of a, so that q (= rk/a) satisfies condition dS]). Since we 
are assuming that 7 < 2\/k, Conjecture |2] is no longer useful. The probability that the rank of an m x 
{k < m) (7, a) regular symmetric banded matrix with 7 < 2^/k is less than k is an open problem for 
an arbitrary choice of r; however, this probability can be given a trivial upper-bound by the probability 
that all the sub-matrices corresponding to different apertures of the underlying regular symmetric banded 
matrix have full rank. The following theorem summarizes the above discussionl^ 

Theorem 11: For any e > 0, an OCC with q chunks of size a, and overlap 7 < 2v^, over any vector 
space F2 , is capacity-achieving for a network of length / with k message packets under any one-in-one-out 
worst-case schedule of capacity n, with failure probability no larger than e, so long as 

k < qip — ql\og{nl/e) — glog(g/e) — ql — q, (6) 

and l^q\n{nl/e) = o{n), where 

if = n/q-0 {{l^{n/qf \xi{ln/ e)Y'^) . 

Furthermore, the larger is the overlap size, the looser would be the upper bound on k. 

Since A;„max = n — (^q{P{n/q)'^ln{ln/e)y^^) —ql\og{nl/e) — q\og{q/e) — ql — q is a decreasing function 
of q (when the other parameters are fixed), the larger is the number of chunks for a given aperture size, 
the smaller is the speed of convergence to the capacity (the ratio knaiax/n- becomes smaller with increasing 
q, for fixed l,n, and e). Thus, similarly as before, an OCC with overlap 7 < 2\/k provides the largest 
speed of convergence to the capacity when its overlap is the smallest possible value. This implies that 

^'Xhe ratio k„,^^/n — [n — 0{q{i^{n/qY\n{ln/e)Y^'^) — qllog{nl/e) — ql — log(l/e))/n is a decreasing function of q. Hence, for 
larger values of q, this ratio becomes smaller when the other parameters I, n and e are fixed, which implies a lower speed of convergence. 

^'Part of the reason that the result of Theorem II 1 lis not tight is that it is based on the assumption that each chunk has to be decodable in 
isolation which is a sufficient but not a necessary condition in the case of overlapping chunks. Indeed, our analysis in this case is sub-optimal, 
and one can expect an OCC with sufficiently large apertures, and smaller overlap to perform even better than what Theorem 1111 presents. 
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among OCC with smaller overlaps, CC has the fastest convergence to capacity for given coding costs. 

All-In-All-Out Worst-Case Schedules: Corresponding to Lemmas [H] and [151 we have the following 
results for all-in-all-out schedules. 

Lemma 16: For any e > 0, applying a dense code to a given chunk tu over a network of length / with 
any all-in-all-out worst-case schedule of capacity n, given that the capacity of the w-schedule is cp, the 
number of dense u;-packets at node t is less than (p — /log(//e) w.p. b.a.b. e. 

Proof: This is a direct result of Lemma |9l by replacing n with (p. ■ 

Lemma 17: Let /, n, q, ip and e be defined as above. Then the node t fails to receive at least tp—l log(g//e) 
dense cj-packets, for every uj, w.p. b.a.b. e, and fails to be larger than 

(l-0(((g/n)ln(/g/e))^/^)).(n/g), 

w.p. b.a.b. e, so long as 

gin — = o {n) . 

Proof: Replacing e with e/g in Lemma [16] (for taking a union bound over q), the first part of the 
lemma follows. Replacing e with e in Lemma [T2] proves the second part of the lemma. ■ 
Lemma [17] along with Conjecture [2] yield the following results. 

Theorem 12: For any e > 0, an OCC with q chunks of size a, and overlap 7 > 2\/k, over any vector 
space F2 , is capacity-achieving for a network of length / with k message packets under any all-in-all-out 
worst-case schedule of capacity n, with failure probability no larger than e, so long as 

k <qh — log(l/e), 

and gln(Zg/e) = o{n), where 

h = n/q-0 (((n/g) \n{lq/e)f''^) - I log(/g/e) - /. 

Moreover, an OCC with the above description but with 7 < 2\/k is capacity-achieving for a network 
scenario as above, w.f.p. no larger than e, so long as 

k < qh — glog(g/e) — g. 
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Coding Costs: The worst-case with regards to the encoding cost at any transmitting interior node u (or 
the source node s) occurs if node u sends all its successful w-packets after receiving all the cj-packets 
sent by the node in the upper layer, and for all i G Xu'^\ Ae,i's (or for all j G M.^^\ /iej's) are chosen 
to be nonzero. The number of packet operations for encoding at any such node u (or node s) is 0{n^ /q) 
(or 0{nQ)). (There are 0{n/q) w-packets sent and received by any such node u and there are 0{n/q) 
w-packets sent by node s.) Thus the encoding cost is 0(a)H^ 

To solve the system of linear equations {y^ = J2jeMs ■ ^ ^ -^*oo } using Gauss-Jordan elimination, 

node t requires 0{kna) field operations and 0{na) packet operations (Qt has a bandwidth and a size of 
a and n x k, respectively). Thus the decoding cost is 0(a), since n/k = 0(1), as n goes to infinity. 

Theorem 13: The encoding and decoding costs of an OCC with an aperture size a are each 0(a). 



E. Comparison 

We now compare CC and OCC with sufficiently large apertures over one-in-one-out worst-case sched- 
ules in the asymptotic regime. Similar results can be shown while comparing these codes over all-in-all-out 
worst-case schedules. 

Consider a one-in-one-out worst-case schedule of capacity n over a network of length /. Suppose that 
the k message packets are divided into rq chunks of size a (= k/q)^ 

We compare Theorem |5] with Theorems [10] and [TT] Note that Theorem |5] for CC is a special case 
of Theorem [TT] where r is set to 1. We study the tradeoff between the probability of failure (referred 
to as the "message error rate" (MER)) and the speed of convergence. In particular, CC and OCC with 
similar aperture size (similar coding costs) are compared with respect to their speed of convergence to 
the capacity when the MER is given. 

Theorem 14: For a given MER and for sufficiently large apertures, OCC with larger overlap has smaller 
speed of convergence than OCC with smaller overlap. 
Proof: It follows from Theorems [10] and [H] that 

knmax/n = 1- (lTq\og(nl/e)/n) , 

^^The number of packet operations for encoding is 0{{l — l)n^/q) + 0{na), and hence the encoding cost is 0{{l — l)n^/qkl) + 0{na/kl), 
i.e., 0((Z — l)n/ql) + 0{a/l) — 0{a), since n/k = 0(1), and k/q = 0{a), as n goes to infinity. 

^''We assume that the number of chunks in OCC with overlap parameter t is rq, not q, as was the case in the previous sections. The 
reason is to be consistent in the definition of the aperture size a as k/q, for both CC and OCC, as we compare CC and OCC with different 
overlap parameters for a similar aperture size. Thus, q needs to be replaced with rq in the results presented earlier for an OCC with overlap 
parameter r. 
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TABLE I 

Comparison of various network codes over line networks with worst-case schedules. 



Network Codes 


Schedule Type 


(n- fc)/fc 


MER 


PER-^ 


Chunk Size (a 




Constraints 


Dense Codes 


One-In-One-Out 
All-In-All-Out 


O |log(i ) 

\ K, v e / / 


e 
e 


- 


k 
k 


- 


CC: Large a 


One-In-One-Out 
All-In-All-Out 


o(^iog(!)) 

0(i log(A)) 


e 
e 


- 


f^(ln(i)) 


- 


CC: Relatively Small a 


One-In-One-Out 
All-In-All-Out 


X 
X 


ek/a 
ek/a 


- 




e:=o(iln(i)) 


CC: Very Small a 


One-In-One-Out 
All-In-All-Out 


X 
X 




e 
e 




A)) 


6 = 0(1) 

e-O(l) 


OCC: Large a 


One-In-One-Out 
All-In-All-Out 


o(^iog(!)) 

0(^l0g(;^)) 


e 
e 




n 
n 


(rln( 
(rln( 


1)) 
^)) 




OCC: Relatively Small a 


One-In-One-Out 
All-In-All-Out 


A 
A 


prk/a 
prk/a 




n 
n 




^ r \ \ 
^ Ac// 




OCC: Very Small a 


One-In-One-Out 
All-In-All-Out 


A 
A 




P 
P 


n 
n 




r \ 

^ Ae/ 
' r \ 

\Af / 




6 = 0(1) 
6 = 0(1) 



for OCC with aperture size k/q, and overlap parameter r, so long as g = o{n / [l^ \\i{nl / e))) . Thus, for 
given n, /, g and e, the larger is r, the smaller is the ratio kn^^^/n. ■ 

Noting that CC is a special case of OCC with zero overlap size, i.e., r is equal to 1, the following 
result is a corollary of Theorem [14] 

Corollary 1: For a given MER and for sufficiently large apertures, CC has higher speed of convergence 
to the capacity compared to OCC with any non-zero overlap. 

Table |I] summarizes the performance of the network codes discussed in this paper over two types of 
worst-case schedules. In particular, the performance measures are the overhead per message, (n — k)/k, 
and the message or packet error rate (MER/PER), for different chunk sizes (a). For example, for each 
type of schedules, the comparison of the corresponding rows of Table U for CC: large a and OCC: large 
a shows that for a given overhead (similar speed of convergence) and a given MER, the chunk size a for 
OCC must be larger than that for CC by a factor of r. This implies that for the same speed of convergence 
and MER, the coding cost of OCC is r times that of CC. Moreover, for both CC and OCC with large a, 
the lower bound on a is a super-logarithmic function of k. 

Therefore, our asymptotic results so far indicate that for sufficiently large apertures, which guarantee the 
convergence to capacity, CC (OCC with zero overlap size) is superior to OCC with any non-zero overlap 
size. One may then wonder about whether there is any provable advantage in using OCC in the asymptotic 
regime. We will answer this question in the next section, and the following provides a motivation. 

^^The parameter p lies in the interval (e-^^^"'^, e^), where x is an arbitrary constant integer sufficiently larger than (r — 1)/A. 
^*The coding costs of the codes are linear in the size of chunks, i.e., 0{a). 
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From Table HI it can be seen that for CC and OCC with large ol to be capacity-achieving (i.e., for 
{n — k)/k to go to zero, as k tends to infinity), the chunk size a needs to be bounded from below by 
a super-logarithmic function of k. However, this lower bound might be larger than the affordable coding 
costs in many practical scenarios. Therefore, in Section |IVl we study CC and OCC with chunks of smaller 
sizes and show that in such cases, OCC can outperform CC in the asymptotic regime (e.g., for relatively 
smaller aperture sizes, comparing an OCC and a CC with similar speed of convergence and coding costs, 
the former has a smaller probability of failure). 

IV. Capacity- Appro ACHING Codes: Towards Linear-Time Network Codes 

Focusing on the design of CC and OCC with smaller coding costs when compared to those with 
sufficiently large aperture sizes discussed in Section Illl in this section, we analyze CC and OCC with 
smaller aperture sizes. 

A. CC with Small Apertures 

One-In-One-Out Worst-Case Schedules: Suppose that a CC with q chunks of size a is applied to the k 
message packets at node s of a network of length / with an arbitrary one-in-one-out worst-case schedule of 
capacity n = (1 + A)/c, for an arbitrarily small but constant A > 0, as goes to infinity. Further, suppose 
that a is selected small enough such that it violates the condition In this case, from Lemma [10] 
follows that for a CC with such a small aperture size a, there exists at least one chunk, say w, so that the 
cj-packets do not form a flow of capacity of at least n/q — O ((/'^(ra/g)^ ln(Zn/e))^/'^) w.p. b.a.b. e. Thus 
Lemma [TOl and Theorems |4]and|5]do not apply to the underlying setting. 

Our approach in this case is to fix a particular chunk, say cu, and give a lower bound on the capacity of 
the w-schedule. We are then able to lower bound the probability of receiving a collection of a innovative 
cj-packets by node t. 

The expected capacity of the cj-schedule is yU = n/q = (1 + \)a. The deviation of the actual capacity 
of the w-schedule from its expectation is upper bounded as follows. 

Lemma 18: For any e > 0, for a CC with aperture size a, over a network of length / with any one-in- 
one-out worst-case schedule of capacity (1 + X)k, for a given u, the capacity of the oj-schedule is less 
than 




(7) 



w.p. b.a.b. e/2, where /i = (1 + X)a. 
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Proof: The proof is similar to that of Lemma [lOl except that we do not need to take a union bound 
over all the chunks. This comes from the fact that in this setting, we need to find the capacity of the flow 
allocated to one chunk alone. ■ 

Lemma 19: For any e > 0, applying a dense code (to a chunk cu) over a network of length / with any 
one-in-one-out worst-case schedule, the number of dense w-packets is not larger than </? — llog{lLp/e) — I 
w.p. b.a.b. e/2, where (p is the capacity of the w-schedule. 

Proof: Note that the actual u;-schedule is not necessarily a one-in-one-out worst-case schedule. 
However, since we are interested in analyzing the worst-case scenario, the w-schedule can also be assumed 
to be a one-in-one-out worst-case schedule. Then, Lemma [19] follows from Lemma |5] by replacing n and 
e with Lf and e/2, respectively. ■ 

Lemma [TSl together with Lemma [19] show that there are less than — IXogilip/e) — I dense w-packets 
w.p. b.a.b. e; and by applying Lemma [6] the probability of decoding failure of a given chunk u can be 
upper bounded as follows. 

Lemma 20: For any e > 0, and A > 0, applying a CC with aperture size a, over a network of length / 
with any one-in-one-out worst-case schedule of capacity (1 + X)k, the probability that a given chunk uo 
fails to be decoded is b.a.b. e, so long as 

a<if-l log(/(^/e) - log(l/e) - / - 1,S (8) 

where the capacity of the cu-schedule is not less than 

(^=(l-0(((/V/x)ln(//i/e))'/'))-/i, 

and ;U = (1 + \)a. 

Proof: Replacing k, h and e in Lemma [6] with a, — llog{l(p/e) — I and e, respectively. Lemma [20l 
follows by a similar argument as in the proof of Theorem [I] except that in this case, we focus on the 
decoding failure probability of a given chunk alone, not all the chunks. ■ 
Substituting (O into ([8]), we obtain 

« = fifSlnf^lV (9) 



'One should note that, despite its appearance, inequality ^ imposes a lower bound on a (the right hand side of inequality ((S} is itself 
a function of q). 
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The details of the derivation of <^ are deferred to Appendix |Wl The above result shows that, over 
any one-in-one-out worst-case schedule, for a CC with an aperture size satisfying (|9]), the probability of 
decoding failure for a given chunk is b.a.b. e. Therefore, the expected fraction of undecodable chunks is 
b.a.b. eg. For sufficiently small choice of e, so long as eg goes to zero, as k goes to infinity, not all the 
chunks would be decodable w.p. b.a.b. eg, and the following is immediate. 

Theorem 15: For any e > 0, when e goes to sufficiently fast, as k goes to infinityo applying a CC 
with aperture size a = /X^) ln(//Ae)), to k message packets over a network of length / with any 
one-in-one-out worst-case schedule of capacity (1 + X)k, all the chunks are decodable w.p. b.b.b. 1 — eg. 

In some cases, however, such small choices of e might not be practical in that the corresponding aperture 
size a (and the coding costs) may be too large (the lower bound on a is a logarithmic function of 1/e). 
Let us assume larger values of e up to a constant. We shall show that for any such e, the actual fraction 
of undecodable chunks is tightly concentrated around its expectation]^ 

Theorem 16: For any e > 0, and A > 0, in a CC with an aperture size a = Q. ((/^/A^) ln(//Ae)), over 
a network of length I with any one-in-one-out worst-case schedule of capacity (1 + \)k, for any 7^ > 0, 
the fraction of undecodable chunks deviates farther than 7^ from e, w.p. b.a.b. e'^'^, for some positive 
constant c = 0(7^e^/a^). 

Proof: Please refer to Appendix |Vl ■ 

Therefore, with high probability, for any 7a > 0, a CC with an aperture size as above fails to recover 
at most (1 + 7a)eA; message packets. This is because the number of chunks is g, and that any undecodable 
chunk accounts for at most k/q unrecovered message packets. 

In such cases, therefore, the expected fraction of undecodable chunks and unrecovered message packets 
are bounded away from zero. Thus, a CC, alone, does not recover all the message packets. Now, a question 
is how a CC with small apertures can recover all the message packets? One solution is to devise a proper 
precoding scheme in order to guarantee the completion of decoding of all the message packets. The 
combination of a CC and a precode is a chunked code with precoding (CCP). 

The precode works as follows. Suppose that the k message packets are the input of a precode of rate 
R. The coded packets at the output of the precode are called the intermediate packets. The number of 
intermediate packets is k/ R. The intermediate packets are sent through the network by using a CC. The 

^^We say that e goes to "sufficiently fast," if tq tends to zero as k goes to infinity, i.e., e/ln(l/e) = o{l/k). 

^'We consider tire worst-case assuming that the probability of decoding failure of any chunk is the largest possible, i.e., e (as shown in 
Lemma I20I1, and hence the expected fraction of undecodable chunks is e. 
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TABLE II 

Comparison of various linear-time erasure-correcting codes 



Erasure Codes 


Coding Costs 


Probability of Failure 


Error-Exponent 


Tomado/LDPC Codes in [20|-[22| 


0(log(l/7p)) 


poly(l/fc) 





LDPC Codes in [23| 


0((l/7p)log(l/7p)) 


poly(l/fc) 





Codes in ^ 


0(l/7^1og(l/7,)) 
(l/7,Mog(l/7p)) 





poly(7p) 



number of the intermediate packets that cannot be recovered at the output of the CC decoder is at most 
pk/R, where p = (l + 7a)e, for any arbitrarily small constant 7a > (as shown in Theorem [16]). Therefore, 
there are at least {1 —p)k/R intermediate packets recovered by the CC decoder. These packets constitute 
the input symbols of the precode decoder. The precode thus needs to recover the k message packets from 
the set of recovered intermediate packets. This implies that the precode has to be an erasure-correcting 
code of dimension k, block length k/R (rate R), that is capable of recovering p fraction of erasures. C 

We are, further, interested in a CCP scheme with linear-time encoding/decoding algorithms (i.e., with 
constant coding costs with respect to k). Therefore, both the CC and the precode must be linear-time 
codes. It should be clear that CC is linear-time so long as the aperture size is a constant, and therefore, 
the problem is to look for a precode with constant coding costsHl 

Table HI] lists a number of linear-time erasure-correcting codes in the literature which provide the best 
known tradeoffs between the computational complexity and the probability of failure. In Table HI] each 
code has dimension k, rate R, and is able to recover a fraction of erasures up to p. The coding costs in 
Table HI] are expressed in terms of the parameter % = (1 — + i?) — 1. 

As it can be seen in Table HI] from top to bottom, for given k and 7p, the speed of convergence of 
the failure probabilities to zero and the coding costs of the codes increase. By choosing R and p to be 
constant, the coding costs will also be constant. For a CCP, the precode needs to recover a fraction of 
erasures up to p = (1 + 7a)e, for an arbitrarily small constant 7a > 0, where e is a constant as well. The 
codes listed in Table HI] are each able to recover a fraction of erasures up to p requiring constant coding 
costs, and thus are each a good candidate to be combined with CC. 

All-In-All-Out Worst-Case Schedules: The results of CC with small apertures over all-in-all-out worst- 
case schedules are quite similar to those over one-in one-out schedules, except that the lower bound on 
the aperture size differs. The difference arises from the difference between the conditions (|2]) and © on 

^"The precode does not have to be capacity-achieving and hence the rate of the precode R does not have to tend to 1 — p. 
^'The specifications of all the precodes discussed in the rest of this paper are the same as those discussed here, and hence not repeated 
for brevity. 
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the aperture size in Lemmas [10] and [I2l respectively. Thus, for brevity, we only state and prove the main 
lemmas; and the resulting theorems would be similar to Theorems \T5\ and [16] (and not repeated), except 
that the lower bound on the aperture size needs to be replaced with a new lower bound derived in the 
following. 

Lemma 21: For any e > 0, for a CC with aperture size a, over a network of length / with any all- 
in-all-out worst-case schedule of capacity (1 + \)k, for a given cu, the capacity of the w-schedule is less 
than 

^=(l-0((ln(//6)//i)^/^))./. (10) 

w.p. b.a.b. e/2, where /i = (1 + X)a. 

Proof: The proof is similar to that of Lemma \T2i except that we do not take a union bound over all 
the chunks. This is because, in this setting, we are interested in the capacity of the flow allocated to one 
chunk only. ■ 

Lemma 22: For any e > 0, applying a dense code (to a chunk co) over a network of length / with any all- 
in-all-out worst-case schedule, the number of dense w-packets is less than y9 — / log(//e) — / w.p. b.a.b. e/2, 
where is the capacity of the w-schedule. 

Proof: In this case, the w-schedule is itself always an all-in-all-out worst-case schedule. Then, the 
result follows from Lemma [9] by replacing n and e with ip and e/2, respectively. ■ 

Lemma [21] together with Lemma [22] show that the number of dense w-packets is less than i^ — l log(//e) — 
1 w.p. b.a.b. e; and the following gives an upper bound on the probability of decoding failure of a given 
chunk u. 

Lemma 23: For any e > 0, and A > 0, applying a CC with aperture size a, over a network of length 
I with any all-in-all-out worst-case schedule of capacity (1 + X)k, the probability of a given chunk cu to 
be undecodable is b.a.b. e, so long as 

a<^ - nog(//e) - log(l/e) - I - 1, (11) 

where 

^=(l-0((ln(//e)//x)^/^))-/i, 

and /X = (1 + X)a. 
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Proof: Replacing k, h and e in Lemma [6] with a, Lp — l\og{l/e) — I and e, respectively, the result 
follows from an argument similar to that of Theorem |2] while focusing on the decoding failure probability 



The details are similar to that of (|9]), and hence omitted. Thus, over any all-in-all-out worst-case schedule, 
for a CC with an aperture size as given in ([12), the probability of decoding failure of a given chunk is 
b.a.b. e. 

B. OCC with Small Apertures 

One-In-One-Out Worst-Case Schedules: Suppose an OCC with q (= kr/a) chunks, each of size a, 
and overlap 7 = a/rg (when Tg = t/{t — 1), and r is a constant integer divisor of a), applied to the k 
message packets at node s of a network of length / with an arbitrary one-in-one-out worst-case schedule 
of capacity n = [1 + X)k. 

The same approach used for the analysis of CC with small apertures is not applicable to OCC with 
small apertures. In particular, unlike CC, in OCC, a martingale argument alone does not provide a tight 
upper bound on the the fraction of unrecoverable message packets. The reason is that in OCC the chunks 
are to be decoded together. (In CC, upper bounding the fraction of undecodable chunks yields a trivial 
upper bound on the fraction of unrecovered message packets as the chunks do not share any message 
packets.) A new approach is thus required for the analysis of OCC. 

In the following, we provide a sketch of our analysis. Consider the decoding matrix Qt at node t, and let 
Q[ be Qt restricted to its dense rows. In the underlying setting, the result of Lemma [TSl is not applicable 
to lower bound the probability that there exists a set of rows in Q'^, so that these rows form a (7, a) 
regular symmetric banded matrix with k columns and a sufficiently large number of rows. Consequently, 
the result of Conjecture |2] (symmetric case) is no longer useful. 

Let X > 1 be an integer sufficiently smaller than the number of chunks. Consider a particular set of 
X contiguous chunks (we will specify the precise choice of x later). Focus on the set of dense packets 
pertaining to the given set of chunks. We first lower bound the probability that the set of rows pertaining 
to these chunks in Q'^ includes a (7, a) regular asymmetric banded matrix with — 7) + 7 columns 



of a given chunk alone, not all the chunks. 



Substituting ([TOl) into ([IB, we obtain 




(12) 
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(the number of distinct message packets in x contiguous chunks) and a sufficiently large number of rows. 
By using Conjecture |2] (asymmetric case), we next upper bound the probability that such a set fails to be 
decodable. Studying all such sets of x contiguous chunks (lower bounding the probability that any such 
set is decodable), the fraction of recoverable message packets can be lower bounded. (All the message 
packets in a chunk belonging to a decodable set of chunks are recoverable). 

It is worth noting that our analysis is sub-optimal in the sense that there might be some recoverable 
message packets that we declare as unrecoverable. This is because the decoding, in our setting, is performed 
on the set of all the chunks together, not on the subsets of chunks in isolation. 

We formalize the above process as follows. We call each set of x contiguous chunks, in an end-around 
fashion, a hyperchunk (the first hyperchunk includes the first x chunks to E and the second 

hyperchunk includes the chunks u E {2,...,x + l}, and so on)o The hyperchunks are overlapping in 
chunks and regardless of the choice of x^ there are q hyperchunks. We also call each (disjoint) set of 
a/r = k/q contiguous message packets a block (the first block starts from the first message packet, the 
second block starts from the message packet next to the last message packet in the first block, and so 
forth). By definition, each hyperchunk consists of^:=x + r — 1 contiguous blocks^j We say that a 
given hyperchunk is not decodable (called a bad hyperchunk) if it fails to be decoded in isolation. We 
also say that a given block is not recoverable (called a bad block) if it does not belong to any decodable 
hyperchunk. 

We shall upper bound the probability that a given hyperchunk fails to be decodable (by lower bounding 
the probability of receiving an innovative collection of x^/q packets belonging to this hyperchunk). 
Lemma [19] serves this purpose when the capacity of the flow by the packets pertaining to any chunk in 
this hyperchunk is given. We lower bound this capacity in the following. 

The expected number of w-packets for a given chunk w is yU = (1 + A)a/r. The capacity of the 
cu-schedule is a random variable and its deviation from the expectation is upper bounded as follows. 

Lemma 24: For any e > 0, A > 0, and any constant integer x > 0, an OCC with an aperture size 
a, and overlap parameter r, over a network of length / with any one-in-one-out worst-case schedule of 
capacity (1 + A)A; fails to provide an w-schedule of capacity larger than 

^= (l-0(((/ V/x)ln(//xx/6)) /X, (13) 

^^We will determine the optimal value of x that results in the tightest possible bounds as part of our analysis. 

^^Since x > 1' ™d in the case of OCC, we have r > 1, then, each hyperchunk contains at least two contiguous blocks (i.e., £ > 1). 
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for a given cu, w.p. b.a.b. e/2x, where /i = (1 + A)a/r. 

Lemma 25: For any e > 0, and any constant integer x > 0, applying a dense code to a given chunk cu 
over a network of length / with any one-in-one-out worst-case schedule, the number of dense cj-packets 
is smaller than ip — / log(/</)x/e) — /, w.p. b.a.b. e/2x, where is the capacity of the w-schedule. 
Lemmas [24] and [25] readily follow from Lemmas [18] and [19] by replacing e with e/x- 
Lemma [24] together with Lemma l25] show that, for a given w, w.p. b.a.b. e/x, there are less than 
— / log(/V9x/e) — / dense cj-packets at node t. Taking a union bound over a set of x chunks (a hyperchunk), 
it can be seen that, w.p. b.a.b. e, there is not a subset of dense packets pertaining to a hyperchunk that 
form a (7,0) regular asymmetric banded matrix of size — 7) + 7) x {x^ ~ log(^V^x/^) ~ xO- 
Then, by applying Conjecture [2l the probability that a given hyperchunk is not decodable can be upper 
bounded as follows. 

Lemma 26: For any e > 0, and A > 0, applying an OCC with aperture size a, and overlap parameter 
r, over a network of length / with any one-in-one-out worst-case schedule of capacity (1 + A)A;, a given 
hyperchunk of size x is bad (fails to be decoded) w.p. b.a.b. e, so long as 

ra<xv- xl^og{l(px/e) - log(l/e) - (14) 

and 7 > TeTy/ra, where 

^= (1-0 (((/ V/i)ln(//iX A)) /i, 

/i = (1 + X)a/T, and r = (x — 1)/t + 1. 

Proof: The result follows from Conjecture [2] by replacing n and k with the number 

(1-0 (((/V/i)ln(//ix/e))'/')) - xnog(//ix/e) - xl 

of rows pertaining to the given hyperchunk that constitute a regular asymmetric banded matrix and the 
number x(a — 7) + 7 = ra of message packets in the given hyperchunk, respectively. ■ 
Substituting into ([H]), we obtain 

by choosing x to be an arbitrary constant integer sufficiently larger than (r — 1)/A. The details are deferred 
to Appendix [VI] 
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Thus, applying an OCC with an aperture size as given in (fTST ). over any one-in-one-out worst-case 
schedule, the probability that a given hyperchunk is bad is b.a.b. e; and thus the expected fraction of 
bad hyperchunks is upper bounded by e. Similarly as before, by using a martingale argument over the 
hyperchunks, it can be shown that with high probability the actual fraction of bad hyperchunks does not 
deviate far from its expected value. 

Theorem 17: For any e > 0, and A > 0, in an OCC with an aperture size a = ((/^/A^)r In ((//Ae)r)), 
and overlap parameter r, over a network of length / with any one-in-one-out worst-case schedule of 
capacity (1 + A)A;, for any 7^ > 0, the fraction of hyperchunks that are not decodable deviates farther 
than 7a from e, w.p. b.a.b. e"'^^, for some positive constant c = 0((7^e^/a^)(Ar)). 

Proof: Please refer to Appendix IVIII ■ 

Now, the problem is to upper bound the fraction of message packets (or blocks) that are not recoverable. 
The fraction of bad blocks however depends on the relative location of bad hyperchunks Q It should be 
clear that among different positioning of bad hyperchunks, the one in which all the bad hyperchunks are 
adjacent results in the largest fraction of bad blocks. Since the expected fraction of bad hyperchunks is 
e, the expected fraction of bad blocks is upper bounded by e. Each block, itself, contains k/q message 
packets, and hence, the expected number of message packets that are unrecoverable is upper bounded by 
ek. 

This, however, is not a tight bound since the probability that a large number of bad hyperchunks are 
adjacent is very small. To derive a tighter bound, the distribution of bad hyperchunks would be required. 
This however is too complex to obtain. We, instead, analyze two extremal types of dependency structures 
of hyperchunks as defined below. 

Let / be the set of integers in [q]. For all i E I, let Qi (Bi) be the set of indices of the message packets 
in the ith hyperchunk (block). We use the same notation Qi (Bi) to refer to the ith hyperchunk (block) 
unless there is a danger of confusion. We, further, let Gi (Bi) be the event that Qi (Bi) is not decodable 
(recoverable). 

Let Gi be the set of events {Gjjiei- Let Jj be an arbitrary non-empty subset of J \ {i}. For any Jj, 
we write i -< U, i y /,/, or i x Jj, respectively, if Pr[Gi| Ajg/. Gj] < Pr[Gi], Pr[Gj| Ajg/^ Gj] > Pr[Gj], 
or Pr[G'j| Ajg/^ Gj] = PrfCj]. For a given probability measure on the set Gj, for any i, and Jj, either 

'''We explain this dependency tliroughi an example. Consider a case with only two bad hyperchunks. Suppose that these two bad hyperchunks 
share a given block which does not belong to any other hyperchunk. This block would therefore be a bad block. Now, consider the case 
where these two bad hyperchunks either do not share any blocks, or any block that they share is in some other hyperchunks as well. In such 
cases, there is no bad block, because each block belongs to at least one hyperchunk which is not bad. 
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i -< li, i >- li, or i X /j. We call the set of relationships {{i -< li) \/ {i y li) V (z x /i)}ig/,/,cA{«}' 
characteristic set of the given probability measure on G/. The set of all probability measures on G/, whose 
characteristic set is the same, is said to have the same type of dependency, and hence any characteristic 
set defines a type of dependency. 

For i E I, let Ng(i) be an ordered set (in an increasing cyclic order) of indices of hyperchunks that 
overlap with the ith hyperchunk, and Jj be an arbitrary subset of / \ {i}. The first dependency type is 
the one that the occurrence of any subset of Gfs, for all j G Ng(i) (j ^ i), increases the probability that 
Gi occurs, i.e., for all li such that li \ Ng{i) = 0, either i y li, or i /«. The second is the one that, 
for all li such that /j \ Ng(i) = 0, either i ^ li, or i >i Li. In the case of both dependency types, for all 
h n Ng(i) = 0, we have either i -< li, or i x /j, and this is, indeed, the case for any possible type of 
dependency between the hyperchunks o For any other Jj, no consideration is made. We refer to the first 
(second) type as the dependency with positively (negatively) dependent neighborhoods. 

We upper bound, for each type of dependency, (i) the probability that not all the blocks are recoverable, 
and (ii) the probability that a block is unrecoverable. Such bounds are "outer" upper bounds for the class 
of dependency structures of the underlying type in that they hold for any dependency structure in the 
class. 

We say that an outer bound is "tight" over the class of dependency structures of a given type, if, in the 
limit of interest (as k goes to infinity, e goes to zero sufficiently fast, or it is bounded away from zero), 
the outer bound is tight for any worst-case structure in the class. 

We derive tight outer upper bounds for each type, by studying the worst case, i.e., given that any 
arbitrary subset of hyperchunks is not decodable, the conditional probability of undecodability of any 
given hyperchunk is the largest possible. For each of the probabilities (i) and (ii), these (tight) outer upper 
bounds act as the two limits of an interval that for any possible type of dependency, a tight outer upper 
bound lies within. We prove the following theorems. (Proofs are provided in Appendix IVIIII) ]^^ 

Theorem 18: For any e > 0, when e goes to sufficiently fastHZl as k tends to infinity, and for any 
A > 0, applying an OCC with aperture size a = fi((/^/A^)r ln((//Ae)r)), and overlap parameter r, over 
a network of length / with any one-in-one-out worst-case schedule of capacity (1 + A)A;, for any type 

^^The undecodability of a subset of hypercfiunks that does not share any chunk with a given hyperchunk Q, increases the probability that 
a chunk in Q has been allocated a sufficiently large number of dense packets. This, therefore, increases the probability of decodability of Q. 

^^Theorems 1 1 81 and 1 191 coiTespond to Theorems 1151 and [T6] respectively. The proofs have similar structure, yet, the bounds on the aperture 
size and the probability of failure of CC are replaced with those for OCC. 

^'For definition, see Footnote 28. 
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of dependency between hyperchunks, there exists a tight outer upper bound on the probability that not 
all the blocks are recoverable. This bound is between e^^'^~^q, and e^q, for arbitrary constant integer x 
sufficiently larger than (r — 1)/A. 

Theorem 19: For any e > 0, and A > 0, for an OCC with aperture size a = ((/^/A^)r ln((//Ae)r)), 
and overlap parameter r, over a network of length / with any one-in-one-out worst-case schedule of 
capacity (1 + \)k, for any type of dependency between hyperchunks, there exists a tight outer upper 
bound on the probability that a block is unrecoverable. This bound is between e^^""^^, and e^, for arbitrary 
constant integer x sufficiently larger than (r — 1)/A. 

All-In-All-Out Worst-Case Schedules: The results of OCC with small apertures over all-in-all-out worst- 
case schedules are similar to those over one-in-one-out worst-case schedules, except that the lower bound 
on the aperture size needs to be revised due to the difference between the conditions Q and (U) on the 
aperture size in Lemmas [T5l and [TTl respectively. For the sake of brevity, in the following, we only present 
the main lemmas along with their proofs. The resulting theorems are similar to Theorems [18] and [191 with 
the only difference being the lower bound on the aperture size, which needs to be modified accordingly. 

Lemma 27: For any e > 0, applying an OCC with aperture size a, and overlap parameter r, to k message 
packets over a network of length / with any all-in-all-out worst-case schedule of capacity (1 + A) A;, the 
node t fails to receive at least — l\og(l/e) — / dense w-packets, for a given w, w.p. b.a.b. e/2, and ip 
fails to be larger than 

(l-0(((l//i)ln(//6))i/2))-/x, (16) 

w.p. b.a.b. e/2, where /i = (1 + A)a;/r. 

Proof: The proof is similar to that of Lemma [TTl yet, no union bound is taken over the chunks since 
we need the capacity of the flow allocated to one chunk only. ■ 
Lemma 28: For any e > 0, and A > 0, applying an OCC with aperture size a, and overlap parameter 
r, to k message packets over a network of length / with any all-in-all-out worst-case schedule of capacity 
(1 + A)A;, the probability of a given hyperchunk of size x to be bad is b.a.b. e, so long as 

ra<xv- xl^og{lx/e) - log(l/e) - (17) 

and 7 > TeTy/ra, where 

y.= (l-0(((l//i)ln(//6))V2)).^, 
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/i = (1 + A)a/r, and r = (x - l)/r + 1. 

Proof: The proof follows from Conjecture |2] similar to that of Lemma [201 by replacing n and k 
with the lower bound given on the number of rows pertaining to the chunks in the given hyperchunk 
which constitute a regular asymmetric banded matrix and the number of message packets in the given 
hyperchunk, respectively. ■ 
Substituting ([H]) into ([JT]), we obtain 

a = 1] ( i-rln f -^r 1 ) , (18) 



by choosing x ^ (r — 1)/A. Over any all-in-all-out worst-case schedule, therefore, for an OCC with an 
aperture size as given in (fTSl ). the probability of decoding failure of a given hyperchunk is b.a.b. e. 



C. Comparison 

Now, we compare our analytical results for CC and OCC with small chunks over one-in-one-out worst- 
case schedules. Similar comparisons are also valid for all-in-all-out worst-case schedules. 

We consider a one-in-one-out worst-case schedule of capacity n = (1 + A)A;, for a given A > 0, over a 
network of length /. Similar to the notations used in Section IIII-E[ let the k message packets be divided 
into rq chunks of size a (= k/q). We partition CC and OCC (with constant overlap parameter r) with 
small chunks into two categories based on their aperture size depending on k, /, r, A and eo We say that 
a code with a = r2((//A)^r ln((//Ae)r)) has "relatively" or "very" small aperture, if erg goes to zero or 
does not, as k goes to infinity, respectively. 

We compare (i) Theorem [15] with Theorem [18] for CC and OCC with relatively small apertures, and 
(ii) Theorem [T6] with Theorem [T9] for CC and OCC with very small apertures. We focus on the tradeoff 
between the probability of failure (MER) or the expected fraction of unrecoverable message packets 
(referred to as the "packet error rate" (PER)) and the speed of convergence. 

Theorem 20: For a sufficiently small given MER and for relatively small apertures, OCC with larger 
overlap has higher speed of convergence than OCC with smaller overlap. 

In the following, for the ease of notation, in the case of OCC, let 6^ be 2, or x + r — 1, for any constant 
integer x sufficiently larger than (r — 1)/A, through the analysis of the dependency types with positively 
or negatively dependent neighborhoods, respectively. Further, in the case of CC, let 6* be 1. 

^^The overlap parameter t of CC is 1, i.e., tiiere is no overlap between the chunks. 
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Proof of Theorem^^ For positive e, e', A and A', and for some positive constants c and c', consider 
two OCC's with aperture sizes a = c(//A)^r ln((//Ae)r) and a' = c'(//A')^r'ln((//A'e')r'), and constant 
overlap parameters r and r' (r' < r), over two networks of length / with two schedules of capacities 
(1 + A) A;, and (1 + \')k, respectively. By the result of Theorem [TSl it can be seen that for a given message 
error rate (i.e., e^rq = e'^'r'q), and for a given aperture size (i.e., a = a'), the speed of convergence of 
OCC with overlap parameter r is larger than that of OCC with overlap parameter r' (i.e, A < A'), so long 



e < 

and c'/c > r/r', for some constant k > 0, where yU = cr/X^, and fi' = c'r' /\'^. ■ 
The following result is a corollary of Theorem |20] 

Corollary 2: For a sufficiently small given MER and for relatively small apertures, OCC has higher 
speed of convergence compared to CC. 

The method of proving the following results are similar to that of Theorem |20] and Corollary |2l and 
hence omitted. The only difference is that, instead of Theorems \T5\ and [T8l Theorems [16] and [19] are 
needed to study the tradeoff between the PER and the speed of convergence. 

Theorem 21: For a sufficiently small given PER and for very small apertures, OCC with larger overlap 
has higher speed of convergence than OCC with smaller overlap. 

Corollary 3: For a sufficiently small given PER and for very small apertures, OCC has higher speed 
of convergence compared to CC. 

These results for CC and OCC with relatively /very small chunks over two types of worst-case schedules 
are also summarized in Table [IJ The comparison of the rows corresponding to CC and OCC with small 
chunks, as shown in Theorems [20] and |2T1 reveals that while r increases, the speed of decreasing MER/PER 
is higher than the speed of increasing the lower bound on a. Thus, for a given a, and for sufficiently large 
A, the MER/PER of OCC with larger overlap size (larger r) are smaller than that of OCC with smaller 
overlap size (e.g., CC). Moreover, for a given constant A, and for relatively small a (i.e., when e goes to 
zero sufficiently fast, as k tends to infinity), the lower bound on a is logarithmic in k. For very small a 
(i.e., when e is a constant with respect to k), however, the lower bound on a is constant with respect to 
k. This implies that for very small a, the coding costs are constant with respect to k, and hence the code 
is linear-time. The above comparison therefore shows that linear-time OCC can outperform linear-time 
CC over networks with worst-case schedules. 



K(n-M')+M'(e/e'-i) 
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V. Simulation Results 

In this section, we present our simulation results in two parts. The first part is concerned about the 
probability that banded random binary matrices are full rank; and the second part is concerned about the 
performance of dense codes, chunked codes and overlapped chunked codes over worst-case schedules. 

A. Banded Random Binary Matrices 

We study both irregular symmetric and asymmetric banded random binary (BRB) matrices. The results 
for regular symmetric and regular asymmetric cases are similar to those for irregular symmetric and 
irregular asymmetric cases, respectively, and hence not presented. 

1) Setup: In symmetric case, we consider matrices with k = 128,256,512 columns, and n = k + m 
rows, where m = 0, 1, 10. For k = 128, 256, we consider the aperture sizes a = k/8,k/4:, k/2, and for 
k = 512, we consider the aperture sizes a = k/lQ,k/S,k/A. In asymmetric case, we consider matrices 
with k = 512, 1024, 2048 columns, and n = k + m rows, where m = 0, 1, 2, 4, 8, 16; and for each k, 
we consider the aperture sizes a = k/8, k/A, k/2. For a given k, the lower bound on the overlap size in 
asymmetric cases is larger than that in symmetric cases, and hence we need to consider larger matrices 
for asymmetric cases. In both symmetric and asymmetric cases, we consider the overlap sizes 7 = a/re, 
where rg = r/(r — 1), for the overlap parameters r = 2, 4, 8. 

We simulate 10000 irregular symmetric or asymmetric BRB matrices for each k,n,a, and 7, and plot 
the relative frequency of the number of times that the simulated matrices have full column-rank (the 
probability that a (7, a) irregular (symmetric or asymmetric) banded random binary matrix of size n x k 
has rank k). For each k and n, for comparison, we also plot the theoretical (asymptotic) result (derived 
in [|25l ) for the probability that a fully random matrix of size nx k has rank k (in a fully random matrix, 
each entry is chosen uniformly at random from F2). 

2) Results: Figures |7l|9] show that for fixed k and n, so long as the overlap size 7 > 2^/k, a (7,0:) 
irregular symmetric BRB matrix behaves similar to a fully random matrix of the same size in terms of 
the probability of being full column-rank. 

Figures [T0lfT2] show similar results for irregular asymmetric BRB matrices. The results show that the 
probability that such matrices have full column-rank is similar to that of fully random matrices when the 
aperture size 7 > T^T^Jk. 
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Fig. 7. The rank property of symmetric BRB matrices: k — 128. Fig. 8. Tiie rank property of symmetric BRB matrices: k = 256. 
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Fig. 9. Tlie rank property of symmetric BRB matrices: k — 512. Fig. 10. The rank property of asymmetric BRB matrices: k — 512. 



B. Dense Codes, CC and OCC: Comparison 

We compare the performance of finite-length dense codes, chunked codes and overlapped chunked 
codes over one-in-one-out and all-in-all-out worst-case schedules. The comparisons are in terms of the 
tradeoff between (i) the overhead per message A := (n — k)/k, simply called "overhead," and the 
probability of decoding failure (message error rate, MER), and (ii) the overhead and the expected fraction 
of unrecoverable message packets (packet error rate, PER). The variables in these comparisons are the 
message, aperture and overlap sizes as well as the network length and the schedule type. 

1 ) Setup: We consider line networks of lengths / = 2, 4. The networks are simulated with random codes 
over one-in-one-out or all-in-all-out worst-case schedules of capacity n = {1 + \)k, for some < A < 3, 
and with the message sizes k = 64, 256. For each k, we consider the aperture sizes a = k/8, k/A, k/2, k. 
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Fig. 11. The rank property of asymmetric BRB matrices: k — 1024. Fig. 12. The rank property of asymmetric BRB matrices: k = 2048. 

and the overlap sizes 7 = a/re, where Tg = t/{t — 1), for the overlap parameters r = 1 (i.e., CC), 2, 
and 4. We are interested in the MER and PER, each as a function of the overhead, for a given aperture 
size (given coding costs). We would also like to investigate how each of these functions changes with the 
aperture and overlap sizes as well as the length of the network and the type of the worst-case schedule. 
For each l,n,k,a and 7, each coding scheme is applied to the underlying networks until 1000 decoding 
failures occur. 

2) One-In-One-Out Worst-Case Schedules: Figures [T3] and [14] depict the MER vs. the overhead A for the 
cases where / = 2, and k is 64, and 256, respectively (for different aperture and overlap sizes). Figures [T5] 
anddH respectively, depict the same scenarios as in Figures [13] and [TH only for longer networks of length 
/ = 4. Similar results for PER, instead of MER, are presented in Figures [T7]l20] Since the trends for MER 
and PER are similar, in the following, we just discuss the MER results. 

For given k and /, investigating the results in each of the Figures [T3][T6] shows that for the same 
aperture size (same coding costs), OCC with larger overlap is more efficient (requires less overhead) than 
CC (OCC with smaller overlap), for sufficiently small MER. More detailed analysis of the simulation 
results is provided in the following. 

The Effect of the Message Size: A comparison of Figures [13] and [14] shows that for a given aperture size 
a, and a given network length, the MER below which OCC (OCC with larger overlap) outperforms CC 
(OCC with smaller overlap) is an increasing function of the message size. So, OCC are more advantageous 
over CC for a given coding cost (a given aperture size), when the message size is larger. For example, 
in Figure \T3i it can be seen that for k = 64, and a = 32, for MERs below about 0.35, OCC with r = 2 
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Fig. 13. CC and OCC over one-in-one-out scliedules: smaller message Fig. 14. CC and OCC over one-in-one-out scliedules: larger message 
size and shorter network. size and shorter network. 
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Fig. 15. CC and OCC over one-in-one-out schedules: smaller message Fig. 16. CC and OCC over one-in-one-out schedules; larger message 
size and longer network. size and longer network. 



requires a smaller overhead compared to CC (r = 1). However, as can be seen in Figure [Ml for larger 
message size k = 256, and similar aperture size a = 32, OCC with r = 2 is superior to CC, for MERs 
below about 1 (for almost all MERs). 

By comparing Figures [15] and \T6[ similar behavior can be observed for longer networks. 

The Effect of the Network Length: Comparison between Figures [l3] and [15] or between Figures [14] 
and [16] demonstrates that OCC (OCC with larger overlap) are more advantageous over CC (OCC with 
smaller overlap) for shorter networks. For example. Figure [l3] (shorter network) shows that for a = 16, 
OCC with r = 2 is superior to CC for MERs below 0.3; however, as can be seen in Figure [T5] (longer 
network), this occurs for MERs below 0.035. 
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Fig. 17. CC and OCC over one-in-one-out scliedules: smaller message Fig. 18. CC and OCC over one-in-one-out schedules: larger message 
size and shorter network. size and shorter network. 



The Ejfect of the Overlap Size: In Figures [T3]fT6l one can see that for a given aperture size a, the MER 
below which OCC (OCC with larger overlap) outperforms CC (OCC with smaller overlap) is a decreasing 
function of the overlap size. Therefore, for a given coding cost, and sufficiently small MERs, OCC with 
larger overlaps are more advantageous. For example, as can be seen in Figure [l4l for a = 32, OCC with 
r = 4 (larger overlap size) outperforms CC for MERs below about 0.6; and OCC with r = 2 (smaller 
overlap size) is superior to CC for MERs below about 1. Moreover, the slope of the curves for OCC with 
r = 4 and OCC with r = 2 shows that the former crosses (outperforms) the latter for a MER somewhere 
below 10~^. 
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Fig. 19. CC and OCC over one-in-one-out schedules: smaller message Fig. 20. CC and OCC over one-in-one-out schedules: larger message 
size and longer network. size and longer network. 



The Effect of the Aperture Size: It can be seen in Figures [T3lfT6l that for a given overlap parameter, and 
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Overhead per Message (X) Overhead per Message iX) 

Fig. 21. CC and OCC over all-in-all-out schedules: smaller message Fig. 22. CC and OCC over all-in-all-out schedules; larger message 
size and shorter network. size and shorter network. 



given message size and network length, the smaller the aperture size (the smaller the coding costs), the 
larger the overhead. 

3) All-In-All-Out Worst-Case Schedules: Figures I2TH241 and |25ll28] represent similar scenarios to those 
in Figures [T3lfT6] and [T7H201 for MER and PER, respectively. While the general trends for all-in-all-out 
schedules are similar to those of one-in-one-out schedules, discussed in the previous part, those trends 
are more pronounced for the all-in-all-out schedules. As an example, comparisons between Figures [13] 
and |2T| reveal that OCC with a = 16 and r = 4 outperforms CC of the same aperture size for MERs 
below 0.3 for one-in-one-out schedules. This crossover MER for the all-in-all-out schedules is improved 
to 0.85. The corresponding values of MER if a is increased to 32 are 0.005 and 0.9, for the two categories 
of schedules, respectively. Therefore, for a randomly generated worst-case schedule, one expects to see 
improvements in the performance/complexity tradeoff of OCC versus CC that are in between those reported 
for one-in-one-out and all-in-all-out schedules. 

Example 1: Consider downloading a 1MB file from a file server 4 hops away (/ = 4). Assuming the 
worst-case scenario, the underlying network is under a worst-case schedule. Suppose that packets of length 
4KB are used for the transmission. This implies that the number of message packets k = 256. Consider a 
target PER of 10^'', and two possible transmission scenarios: (a) using a CC with a = 64, and (b) using 
an OCC with a = 64 and r = 2 (7 = 32). From Figure |20](for one-in-one-out schedules), one can see 
that the overhead A for the two scenarios (a) and (b) is about 0.85 and 0.7, respectively. From Figure l28l 
(for all-in-all-out schedules), it can be seen that the overhead A for the two scenarios is about 0.5 and 
0.35, respectively. This implies that downloading the file by scenario (b) is from about 17% to about 30% 
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Fig. 23. CC and OCC over all-in-all-out schedules: smaller message Fig. 24. CC and OCC over all-in-all-out schedules; larger message 
size and longer network. size and longer network. 
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Fig. 25. CC and OCC over all-in-all-out schedules: smaller message Fig. 26. CC and OCC over all-in-all-out schedules: larger message 
size and shorter network. size and shorter network. 

faster than scenario (a), depending on the type of the worst-case schedule under consideration. 



VI. Conclusion 

In this paper, we analyzed and designed communicationally and computationally efficient network codes 
over line networks with worst-case schedules. 

We first presented a detailed analysis of random linear network codes (dense codes). This analysis and its 
building blocks then served as part of our machinery to analyze chunk-based coding schemes. In particular, 
tighter bounds on the performance of "chunked codes" (CC) by Maymounkov et al. were derived. It 
was shown that for sufficiently large chunks (super-logarithmic aperture size in the message length), 
chunked codes asymptotically achieve the capacity, yet with a slower speed of convergence (smaller 
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Overhead per Message (X) Overhead per Message iX) 

Fig. 27. CC and OCC over all-in-all-out schedules: smaller message Fig. 28. CC and OCC over all-in-all-out schedules; larger message 
size and longer network. size and longer network. 

communicational efficiency) compared to dense codes. This is the price for the superior computational 
efficiency of CC. 

Searching for codes that are more computationally efficient than dense codes and more communica- 
tionally efficient than chunked codes, we proposed and analyzed chunked codes with overlapping chunks, 
called "overlapped chunked codes" (OCC). 

We showed that (i) for sufficiently large apertures (super-logarithmic in the message length), an OCC 
with a constant overlap parameter is asymptotically capacity-achieving, and (ii) for a given aperture size 
(similar coding costs), an OCC with sufficiently large apertures performs inferior to a CC (OCC with 
zero overlap) with regards to the tradeoff between the speed of convergence and the message error rate. 
We also proved that the larger is the overlap size in OCC with sufficiently large apertures, the worse is 
the tradeoff between the speed of convergence and the message error rate. 

Targeting practical scenarios, where computational resources are scarce, we analyzed CC and OCC 
with smaller aperture sizes (logarithmic or sub-logarithmic in the message size down to some constant). 

We showed that there exists a lower bound on the aperture size logarithmic in the message length, 
such that a code with relatively small apertures satisfying the given bound, yet not satisfying the bound 
for sufficiently large apertures, asymptotically approaches the capacity with an arbitrarily small constant 
non-zero gap. We also proved that for codes with relatively small aperture sizes, the larger is the overlap 
size, the better would be the tradeoff between the speed of convergence and the message error rate. 

We, next, showed that there exists a lower bound on the aperture size constant in the message length. 
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such that a code with very small apertures, i.e., with an aperture size satisfying the given bound, but 
not satisfying the bound for relatively small apertures, asymptotically approaches the capacity with an 
arbitrarily small constant non-zero gap. In this case, the message error rate is bounded away from zero, 
yet still, the packet error rate can be made arbitrarily small. We also proved that for codes with very 
small aperture sizes, the larger is the overlap size, the better would be the tradeoff between the speed 
of convergence and the packet error rate. In fact, one of the main contributions of this work was to 
design linear-time network codes (OCC with very small apertures) that are both computationally and 
communicationally efficient, and that outperform linear-time CC. 

In line with the asymptotic analytical results, we also presented finite-length simulation results which 
verified that OCC with larger overlaps are more efficient, both communicationally and computationally, 
when sufficiently small message or packet error rates are desired. 

Appendix I 
Proof of Lemmas [I]-l9] 

Proof of Lemma [U- Let v = Tu, and v = [vi, ...;vh]. The proof of uniformity of Vi's is straight- 
forward. To prove the independence of Vi's, it suffices to show that for any integer I < j < h, vj is 
independent of the set of fm's, for all 1 < m < j. First, the independence of V2 and vi. The vector of 
coefficients in the linear combination of vi is linearly independent of that of t>2, due to the assumption 
of T having full row-rank. Thus there exists at least one entry of u which only appears in either vi, or 
V2. Since the entries of u are all independent, this implies the independence of V2 and vi. Second, the 
independence of and V2,vi: The following lemma shows that v-^ is independent of V2,vi, if and only if 
Vs is independent of any non-zero linear combination of V2 and Vi, given that V2 and vi are independent. 

Lemma 29: [|26l A discrete random variable z is independent of a set of independent Bernoulli random 
variables if and only if z is independent of any non-zero linear combination of the Bernoulli random 
variables with coefficients from F2. 

Since vi and f 2 are independent, then by Lemma |29l it suffices to show that V3 is independent of any 
non-zero linear combination of V2 and Vi. The vector of coefficients in the linear combination of V3 is 
linearly independent of those of V2 and vi, and hence is linearly independent of any non-zero linear 
combination of V2 and Vi. Then, for any given non-zero linear combination t;i 2 of V2 and vi, there exists 
at least one entry of u, say Ui, for some 1 < i < h, so that U£ only appears in either or t>i 2. Thus, 
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and f 1 2 are independent. Similarly, it can be shown that for all I < j < h, Vj is independent of the set 
of I'm's, for all 1 < m < j, as was to be shown. ■ 

Proof of Lemma^- By the assumption of r(T) > h, there exist at least h linearly independent rows 
in T. Let T' be T restricted to these h linearly independent rows in T, i.e., T' is an h x d sub-matrix of 
T. Then, T'M has h rows which are a subset of rows in TM. Clearly V{TM) > V{T'M). For i < h, 
and j < k, (T'M)ij = E1=i ^,mMn,i and {M„j : m < d,j < B are all i.u.d. Then, for < h, 

< k (and for i ^ i', or j ^ j'), {T'M),j and {T'M)i>^j> are i.u.dO Thus the h rows in T'M are all 
dense, i.e., V{T'M) = h, and hence V{TM) > h. U 

Proof of Lemma [3} For any integer < 7 < d — 1, let T' be T restricted to its first d — 7 columns, 
i.e., T' is an n X (rf — 7) sub-matrix of T. Suppose that there exists a non-zero column-vector u of length 
{d — 7) whose entries are from F2 and that the column-vector T'u of length n is an all-zero vector. This 
is the necessary and the sufficient condition for r(T') < c? — 7. Suppose that the first non-zero entry of u 
is the jth entry. There are 2'^'^~^ such vectors. Since there exist at least d — j + 1 i.u.d. random entries 
in jth column of T', there exist at least d — j + 1 i.u.d. random entries in the vector T'u. The probability 
of all these entries being zero is 2~^'^~^~^^\ and thus the probability of T'u being zero given that it is a 
vector with the first non-zero entry being the jth is at most 2~^'^^^'^^\ Taking a union bound over all such 
vectors u whose first non-zero entry is the jth, the probability of T'u being an all-zero vector given that 
the first non-zero entry of u is its jth is at most 2'^"'^"^ x 2^^'^^^^^^ = 2"'^'^"'^^). Taking a union bound over 
all j, (i.e., 1 < j < (i — 7), the probability of T'u being an all-zero vector is at most {d — 7)2"^'''+^^ ■ 

Proof of Lemma |4l- Since is a x V{Qu) matrix over F2 such that for all j G X„^, |T'^^.^J > 
T^iQu) - j + 1, Lemma [3] implies that r{T'^) > V{Qu)-i, for any integer < 7 < V{Qu) - 1, fails to hold 
w.p. b.a.b. {V{Qu) - 7)2"^-^ Lemma [2] together with Lemma |3] give V{Q,) = V{T^Q'J > V{Qu) - 7 
fails to hold w.p. b.a.b. (V{Q^) - -f)2~^'~\ Setting {V{Qu) - 7)2-^^1 < e, we get 

7 > log(P(QJ - 7) + log(l/e) - 1. (19) 

Let 7 be the smallest integer equal to or larger than logV{Qu) + log(l/e)o Thus, ^'(Q^) > T>{Qu) — 
\ogV{Qu) — log(l/e) fails to hold w.p. b.a.b. e. ■ 

^'For j 7^ j' , {T' M)ij and {T' M)i'ji are linear combinations of disjoint sets of i.u.d. random variables over F2 and hence they are 
i.u.d.; and for j — j' and i 7^ i', {T'M)ij and {T'M)ii j are linear combinations of the same set of random variables over F2 with linearly 
independent vectors of coefficients, and hence they are i.u.d. (by Lemma [TJ. 

""Such a choice of 7 satisfies ^ because for any 7 > 0, log2?(Q„) + log(l/e) > log(0(Q„) - 7) + log(l/e) > log{V{Qu) - 7) + 
log(l/e) - 1. 
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Proof of Lemma^ Since V{Q^) < n for every node v, replacing e with e/(/ — 1) in Lemma HI we 

get 

V{Qu)-V{Q,) < logP(g„) + log((/-l)/e) 

< log(n(/ - l)/e) 

< log(ra//e), 

which fails to hold w.p. b.a.b. e/(/ — 1). Thus, 

i-i 



{mvj - m^^^J) <(/-!) log(n//e) 



i=l 



fails to hold w.p. b.a.b. e. Further, 



i-i 



Since V{QyJ = n, we get 



i=l 



l-l 



1=1 

> n- (/- l)log(n//e) 

> n — /log(n//e) 

which holds w.p. b.b.b. 1 — e. ■ 

Proof of Lemma ^ Clearly, r(M) < A; if there exists a non-zero column vector u of length k, so 
that Mu = 0. For a given non-zero u, the probability that Mu = is equal to 2"*^. Taking a union 
bound over all such vectors u, Pr[r(M) < k] is upper bounded by (2^ — 1)2~^ < 2^"'^. Setting 2^~'^ < e, 
i.e., k <d- log(l/e), we get Pr [r(M) < A;] < e. ■ 

Proof of Lemma ^ The following lemma is useful in order to give an upper bound on Pr [r(T) < 
d — 7], for any integer < 7 < d — 1. 

Lemma 30: ||25l Theorem 1] Let T be a n x d matrix over F2 whose entries are all i.u.d. random 
variables. Then, for any integer 1 < 7 < d — 1, 

Pr[r(T) = d - 7] = . 2-t("-'^+t), 
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for c, = TC„„,+,+i (1 - 2-0/nj=i (1 - 2-0- 

For any integer 1 < 7 < ci — 1, it can be shown by induction that < 1? . Thus, 

Pr[r(T) = - 7] < 2^ • 2-^("-'^+^) < 2^ • 2"^' < 2"^. 

The probability of r(T) < (i — 7 is the sum of the 
or 1. Thus, 

Pr[r(T) < C/-7] = 

< 

■ 

Proof of Lemma^ Since is a n x V{Qu) matrix over F2, such that for all j G X^^, |T'|/2|I = 
Lemma |7] implies that r(T^) > 'D{Qu)—'j fails to hold w.p. b.a.b. 2^^, for any integer < 7 < V{Qu) — 1. 
Lemma [2] together with Lemma |7] then result in V{Qi,) = D(T^Q^) > V{Qu) — 7, which fails to hold 
w.p. b.a.b. 2^^. Setting 2^'^ < e, we get 7 > log(l/e). Let 7 be the smallest integer equal to or larger 
than log(l/e). Thus, > V{Qu) - log(l/e) fails w.p. b.a.b. e. ■ 

Proof of Lemma HI' The proof follows the same steps as the proof of Lemma |5l except that the 
bound of Lemma H] is replaced by that of Lemma [8] ■ 

Appendix II 
Proof of Lemma [TOl 

We show that with high probability for any uj, the most of w-packets in any given subset of packets 
consecutively received by a node (when the size of the subset is chosen with some care) are matched up 
with the most of cj-packets in a particular subset of packets consecutively transmitted afterwards by the 
node. We formalize this idea in the following. 

Consider a non-sink (or non-source) node u (or v). Consider the transmissions over the link (u, v). We 
partition the n packets sent (received) by the node u (node v) into h departure {arrival) buckets from 
the perspective of the node u (node v) such that the first bucket includes the first n/h packets sent (or 



probabilities of r(T) being d — (7 + 1), d— (7 + 2), 



^ Pr[r(r) = d 

1=7+1 
d-l 

i=7+l 

9-7 _ 2-('^-i) 
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received) by the node, the second bucket includes the second n/h packets and so forth. The packets sent 
by the node u might not be received in order by node v, and hence, for all 1 < j < 6, the packets in the 
jth departure bucket of node u might be different from those in the jth arrival bucket of node v. 

The expected number of w-packets in any departure or arrival bucket is jj, = n/hq, for any divisor hq 
of n, and the expected number of w-packets in all the departure or arrival buckets is n/q. The chunks are 
however randomly scheduled by the nodes and hence the actual number of w-packets in any bucket is a 
random variable. 

Let Y denote the number of cj-packets in a given bucket. Thus, E[F] = ji. The set of labels of packets 
in the bucket is denoted by /. Thus, |/| = n/h. Every packet in the bucket happens to be an u;-packet 
independent of any other packet. Random variable Y is thus the sum of independent indicator random 
variables {Xi : i E 1} where Xi is 1, if the ith packet is an w-packet and, Xj is 0, otherwise. 

Lemma 31: [|27l Corollary A. 1.1 4] Let Y be the sum of mutually independent indicator (arbitrarily 
distributed) random variables, and /i = E[F]. For any 6 >0, Pt[Y > (1 + 6)fi] < e-^'^/^ and Pr[y < 
(1 - < e-^"''/^ 

Let /i', and /i" be /i — c^/JI, and yu + Cy/JI, respectively, for some positive c. Taking 5 = c/ y/JI, and 
c = y/2 ln(2/e) = 0(^ln(l/e)) in Lemma [HI we get Pr[r > fi"] < e/2, and Pr[F < fx'] < e/2, i.e., Y 
is less than f^i', or greater than i^i", w.p. b.a.b. e. Thus the number of w-packets for a given w in a given 
bucket fails to lie within /i' and fi" w.p. b.a.b. e. 

We are interested in deriving an upper bound (or a lower bound) on the number of unusable (or usable) 
cu-packets for a given u. 

Fix a particular interior node u {u = Vi, for 1 < i < I). There exists at least one packet sent by node 
u at some time r' > r, after receiving a packet at time r. Thus, all the packets in the jth arrival bucket 
of node u, for all I < j < b, land at node u before any of the packets in the (j + l)th departure bucket 
of node u leaves. Thus, a given cj-packet in the jth arrival bucket of node u can be matched up with any 
w-packet in the (j + l)th departure bucket of node u. 

Consider the w-packets over two tandem links (w, u) and (m, v) (note that node w or node v might be 
node s or node t, respectively). We randomly choose /i' w-packets in each departure (arrival) bucket of 
nodes w and u (nodes u and v) and call them half- good from the perspective of node w, or node u (node 
u, or node v), respectively. Since each packet over these two links belongs to one departure bucket (of 
node w, or node u) and one arrival bucket (of node u, or node v), we call a packet good if it is half-good 
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from the perspective of both nodes w and u (or nodes n and v). 

For all 1 < j < b, there exists at least one particular good w-packet in the (j — l)th arrival bucket of 
node u to be matched up with any given good w-packet in the jth departure bucket of node u. Thus, the 
set of good cj-packets in all but the last arrival bucket of node u are a subset of usable w-packets received 
by node u and hence the number of good cj-packets in these buckets gives a lower bound on the number 
of usable w-packets at node u. 

Taking c = o{yfJi), the number of those cj-packets in any given bucket of any node which are not good 
fails to be less than ji" ■ 0{c/ ^JJi) = 0{c^JJi) w.p. b.a.b. ecj Taking a union bound over all but the last 
arrival bucket of node m, w.p. b.a.b. {h — l)e, the number of w-packets which are not good fails to be less 
than (6 - 1) ■ O(c^) < b ■ 0{c^) = 0{bc^/^^) = Oic^f^q). 

The number of w-packets which are not usable due to landing at the 6th arrival bucket of node u is, 
w.p. b.a.b. e, larger than /i" = /i + c^Ji = (1 + o(l))yU = 0(/i) = 0{n/hq). Such packets are unusable 
because there is no departure bucket at node u, such that the packets therein can be matched up with the 
packets in the last arrival bucket of node u. 

Thus by adding the number of packets in both groups of unusable packets together, i.e., 0{c^Jnb/ q). 



and 0{n/bq), the probability that there are more than 0{cynb/q + n/bq) unusable w-packets at node u is 
at most be. Summing over all the interior nodes of the network (t;j's for all 1 < i < /), w.p. b.a.b. {l — l)be < 
Ibe, the total number of unusable w-packets at node t fails to be less than 

0{{l - l){c^/^ihjq + n/bq)) < 0{l{c^^^ + n/bq)). (20) 

We now specify c so that for every u the probability that the number of unusable w-packets at node 
t is larger than (lIOl l is at most e. Replacing e by e/lbq in Lemma [3T1 one can readily see that c needs 



to be at least 0{\J\n (Ibq/e)). Let c = ©(a/Ih (In/e)) (bq is a divisor of n and hence smaller than n). 
Minimizing the number of unusable w-packets at node t with respect to b (by setting the derivative of 

Hereafter, we operate under the assumption that the actual number of oj-packets in any bucket lies within fi' and fi" . Let fi* be the actual 
number of oj-packets (for a given oj) in a given bucket. We randomly choose /i' packets from the set of fi* packets (i.e., /i* — n' packets will 
not be chosen). Therefore the probability that a given oj-packet in the given bucket is not half-good from the perspective of its transmitting 
node is (/x* — ^')/fi*. Since /i* < /i" , the latter probability is upper bounded by {fi" — fi')/fi". Similarly, from the perspective of its 
receiving node, the given oj-packet fails to be half-good w.p. b.a.b. {/j." — Thus the probability that a given oj-packet is not good (it 

is not half-good from the perspective of its transmitting or receiving node) is at most 2{fj," — fJ.')/fJ," < 2(/i" — fJ-')/ fJ-' ~ 4c/(y^— c) (the 
inequality follows from n' < jj," , and the equality follows from replacing /i' and fi" , respectively, with jj, — c^/Jl, and /i + c^fji). Taking 
c = o(^), we have 4c/(^ - c) < 4c/[(l - o(l))^] < (1 + o(l))(4c/^) = 0(c/Vm). 
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(|20l) with respect to b equal to zero and solving for b), we obtain 



n 



1/3 



q\n{ln/e) 

where the ceiling [■] has no significant effect provided that q\n(ln/e) = o{n). Such choices of c 
and b ensure that c = o{^/Jl) = o{^/n/bq), because o{^Jn/bq) = o((n/g)^/^ ln'^/^(/n/e)), and c = 
0(v/ln(/n/e)) = o{{n/qY/Hn^/\ln/e)), so long as q\n(ln/e) = o{n). Substituting c and b into (|20|) . the 
total number of unusable cj-packets at node t is at most 

O (l {nlqf\n^'\ln/e) 



w.p. b.b.b. 1 — e. The expected number of w-packets received by node t is n/q. The actual number however 
is lower bounded by bfi' = n/q — 0{{n / q^/^ \n^^^ {In / e)) . The capacity (/? of the cj-schedule, for every 
uj, (i.e., the number of usable cj-packets at node t) is therefore lower bounded by subtracting the number 
of unusable cj-packets at node t from the total number of w-packets at node t, i.e., 



0{{l\q/n)Hln/e)Y"]].{n/q) 



w.p. b.b.b. 1 — e, so long as 



t3 1 f \ 

I qm — = o[n) 
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The last condition is only needed to ensure that the number of unusable cj-packets at node t is asymptot- 
ically smaller than the total number of w-packets at node t. 



Appendix III 
Proof of Lemma [I2] 

By applying Lemma [3T1 it can be easily shown that, for a given to, the number of w-packets over a 
given link fails to be larger than 

[l-0[{{q/n)Hl/e)f')).{n/q), (21) 

''^Assuming l'^qln{ln/e) = o(n), one can easily conclude that 0{{l'^{q/n)ln(ln/e))^^'^) is asymptotically o(l) with respect to n. 
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w.p. b.a.b. eo To lower bound the number of w-packets for all the chunks, and over all the links, we take 
a union bound over the number of links and the number of chunks by setting e = e//g in (|2T1) and (|22l) . 
yielding Q and dH), respectively. 



Appendix IV 
Details of Equation ^ 

Inequality ([8]) can be rewritten as 

+0{l\og{lip/e)) 
+0(log(l/e)). 

By replacing fi with (1 + A)a, the latter inequality reduces to 

a = fi((//A)a'/3 \n^'\al/e)) 
+fi((//A)logM/e)) 
+fi((l/A)log(l/6)). 

The first term in the above relationship dominates the other two, and hence we obtain 



(23) 



which results in 



a = fi((/7A^)ln(//Ae)). 



''^For a given chunk uj, let Xi, 1 < i < 7i, be an indicator random variable, so that Xi is 1, if the ith packet is an tj-packet, and Xi 
is 0, otherwise. Since each packet pertains to a randomly (uniformly) chosen chunk, Xi's are independent Bernoulli random variables with 
Pr[Xi = 1] = 1/q. Let Y be the number of (^-packets over a given link, i.e., Y = X]i<i<n Then, E[Y] = n/q. By Lemma [STl for 



any 5 > 0, Pt[Y < (1 - 5)n /q] < e"^ "Z^ ''. Setting e"'''"/^'' = e, we get 5 = y^2{q/n) ln(l/e) = 0{^{q/n) ln(l/e)), and hence we 
can write Pr[y < (1 — 0(\/ [q/n) ln(l/e))) • [n/q)] < e. For 5 to be in the range (0, 1) (for Chemoff bound to be valid), we need the 
condition 

g ln(l/e) = o(n) . (22) 
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Appendix V 
Proof of Theorem [16] 

Before giving the proof of Theorem \T6\ we provide a brief overview of matringales, which are useful 
in proving some concentration results (see, e.g., [|27l Chapter 7]). 

Let L : A'^ — M be a functional in a probability space = A^, where Vt denotes the set of functions 
f : B ^ A. We define a measure by setting the values of Pr[/(6) = a], where the values f{h) are assumed 
to be mutually independent. We also fix a gradation {Sjj^Q, i.e., 

= i3o c i5i c ■ ■ ■ i3„ = 

We define a martingale Xq.Xi, ...,X„i by setting 

X,{h) = E[L(/)|/(6) = h{b) for all b e B,], 

where h is a function in f2. Xq is a constant, the expected value of L of the random /. X.^ is L itself. We 
say the functional L satisfies A-Lipschitz (or Lipschitz) condition relative to the gradation so long as for 
some < z < m, if /i and h' differ only on B,+i \ Bi, then \L{h') - L{h) \ < A (or \L{h') - L{h)\ < 1). 

Let L satisfy A-Lipschitz condition, and fi = E[L(/i)]. Then, for any 7^ > 0, Azuma's inequality states 
that 

Pr [L{h) > (1 + 7,)/i] < e-T'^'/2™^. 

Proof of Theorem [T6l' We define the sequence of n (= (1 + A)A;) independent random variables 
hi, ...,hn (denoted by vector h) such that hi represents the index of the chunk to which the zth packet 
received at node t pertains. We define L{h) as the number of not fully decodable chunks given a specific 
vector h. The sequence of Xq, Xi, X^ defined as = 'E[L{h)\hj for I < j < i] yields a standard 
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martingaleC 

Further, the functional L, as defined above, has Lipschitz property in that the number of chunks that 
cannot be decoded can differ by at most one if two sets of the indices of the chunks to which the n packets 
(at node t) pertain differ in only one index. The expected number of chunks that cannot be recovered is 
eg, and applying Azuma's inequality, by setting m = {1 + X)k, and n = eg, for any 7^ > 0, we can write 



Pr[L(/i) > (1 + 7,)eg] < e 



Appendix VI 
Details of Equation (fTSl ) 

Inequality (fT4)) can be rewritten as 

+0{llog{lx^i/e)) 
+0{{l/x) log(l/e)). 

By replacing fi with (1 + A)a/r, and letting p = (xA + 1) /r — 1, we can write the inequality as 

a = n{l{x/p){a/Tf/Hn'/'{lxa/Te)) 
+n{l{x/p)\og{lxa/Te)) 
+fi((l/p)log(l/e)). 

''^Here, Bi (for < i < n) is defined as hi = {hj}j-i. We shall show E[Xi+ijXi] = Xi, where Xi — {Xj}j-Q. Using the independence 
of hj's, we can write 

E[Xi+i\Xi] = E[E[L(/i)|B,+i]|X,] 
E[E[L{h)\B,+i]m 



{<=) 



(d) 



J2 Pr[B,+i\B,]E[L{h)\B,.+i] 



Pr[e,+iie.] ^ Pr[Br,\B,+i]L{h) 

8.+ l\e. S„\8.+i 

^ Pr[Bn\B^]L{h) 
= E[L{h)\B^] 

where (a) since Xi+i = E[L{h)\Bi+i], (b) Xi is constructed based on Bi, (c) since Pr [E[L(/i)|Si+i]|Si] — Pr[Si+i|Si], (d) by expanding 
E[L{h)\Bi+i], and (e) by using Bayes' rule. 
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In the above relation, the first term dominates the other two, and we thus have 

a = n{{l^x^/{p\^)) Hlxa/re)). (24) 
For X ^ (t — 1)/A, we have p ~ xA/r, and (124)) will be equivalent to 

a = Q{{iyX^)T ln{lxa/Te))n (25) 
Condition (|25l) is satisfied if we have 

and 

The first condition is met when 

a = fi((/VA^)rln((//A)r)). (26) 

Choosing x to be a constant integer sufficiently larger than (r — 1)/A, the second condition can be rewritten 
as 

a = fi((/VA=^)rln((//Ae)re)). (27) 
Thus, both conditions (|26l) and (1271 ) are met when 

a = Q{{iyX^)Tln{{l/Xe)r)). 

Appendix VII 
Proof of Theorem [TtI 

Let n = (1 + A)A;, and the sequence of independent random variables hi,...,hn, be such that hi 
represents the same random variable as defined in the proof of Theorem [T6l Let L{h) be the number of bad 
hyperchunks given a specific h (the vector of n outcomes /i/s). As in the proof of Theorem [T6l a martingale 
sequence can be constructed based on L. However, unlike before, here, L has x-Lipschitz property in that 
if two sets of output symbols differ in only one symbol, then the number of bad hyperchunks can differ 

''^For X ^ {''' ~ 1)/-^' we have p ~ 0, and ( I24t will be equivalent to a = fi((?''/A'Vp^)re ln(/rea/Ae)). Yet, for such a choice of x> 
since p ~ 0, the lower bound on a is much larger (and not desirable) compared to that in ( 125) . 
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by at most x (each output symbol pertains to one chunk, and each chunk belongs to x hyperchunks). The 
expected number of bad hyperchunks is eq, and applying Azuma's inequality gives 

Vi[L{h) > (1 + 7a)eg] < e-(^'^'/2a2)("'/'^)^ 

for any 7^ > 0. 

Appendix VIII 
Proof of Theorems [T8] and [T9] 

In the following, we study the two types of positively and negatively dependent neighborhoods. Putting 
the following results together. Theorems [TS] and [19] are immediate for any type of dependency between 
hyperchunks. 

Positively Dependent Neighborhoods: The worst case in this type occurs when for all i E I, and 
all subsets U of I \ {i}, so that k n Ng{i) = 0, Fi[G.i,\ = Pr[G'i], and for all other J^'s, 

For all i E I, let A^e(z) be an ordered set (in an increasing cyclic order) of indices of the hyperchunks 
that overlap with the zth block. Each block is bad if the hyperchunks including it are all bad. Thus, 
for all i E I, Pr[i?j] = 'P^[AjeNs{i)^j^- ^ given i E I, consider a permutation {ki,...,ki} of the 
elements of the set N^^i), so that ki ^ Ng{k2). Such a permutation always exists as \Ns{i)\ > \Ng{kj)\, 
for all I < j < i. Thus, Pr[Gfe2l'-^fci] = Prl^fej]- It can also be verified that for such a permutation, 
the hyperchunks {Qj : j E {k^, ke}} overlap with at least one of the hyperchunks Qkj and Qk2- The 
probability that a hyperchunk fails to be decodable is shown to be b.a.b. e, and hence, for all k E NQ{i), 
Pr[Gfc] < e. By the above argument along with applying chain rule, we have Pr[i?i] = Pr[AjeArB{j) ^j] = 
PrfCfcJ Pr[Gfe|GfeJ = Fi[G,,] Fi[G,,] < := ^ 

Each block is bad w.p. b.a.b. ^, and for any choice of e, so long as ^ goes to zero sufficiently fast with 
k, by applying a union bound, one can show that not all the blocks are recoverable w.p. b.a.b. ^q. The 
tightness of this upper bound can be readily shown by rewriting Pi'[AjeAf8(i) ^j] as 1 — Pi[\/ j^^^^^^^-^ Gj], 
and using the inclusion-exclusion principle. 

Since the expected fraction of bad blocks is upper bounded by ^, for larger choices of e up to a constant, 
by constructing a martingale as before - yet this time, over the blocks, not the hyperchunks - we would be 
able to prove the concentration of the actual fraction of bad blocks around its expected value as follows. 
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Theorem 22: For any e > 0, and A > 0, in an OCC with an aperture size a = Vt ((Z^/A^)r In ((//Ae)r)), 
and overlap parameter r, over a network of length / with any one-in-one-out worst-case schedule of capac- 
ity (1+A)A;, for any type of dependency between hyperchunks with positively dependent neighborhoods, for 
any 7^ > 0, the fraction of blocks that are not recoverable deviates farther than 7^ from ^, w.p. b.a.b. e~^^, 
for some positive constant c = 0((7^^^/a^)(Ar)). 

Proof: Let £ be the number of blocks that a hyperchunk contains, i.e., x + t — 1- The proof is similar 
to that of Theorem [l7] except that L is ^-Lipschitz in this case. The expected number of bad blocks is 
^q, and applying Azuma's inequality, for any 7^ > 0, we have 

Pr[L(/^)>(l+7,)eg]<e-=^ 

by choosing c = O^^-iK^ / a^){T^ / 1)). ■ 
Each block has k/q message packets. Therefore, the result of Theorem 122] shows that for any type 
of dependency between hyperchunks with positively dependent neighborhoods, w.h.p., for any arbitrarily 
small constant 7^ > 0, the number of message packets that are not recoverable is upper bounded by 

Negatively Dependent Neighbors: The worst case in this type occurs when for all i E I, and all 
subsets /j of / \ {i}, FT[Gi\ Gj] = Pr[G'i]. That is, for all i E I, G/s are independent. Thus, 

Now, for any choice of e, so long as t] goes to zero sufficiently fast with k, by applying a union bound, 
one can see that not all the blocks are recoverable w.p. b.a.b. rjq. We prove the tightness of this upper 
bound in the following. 

Suppose the problem of lower bounding the probability that a set of events Bi's do not occur. For mutu- 
ally independent events, Pr[/\.g^ Bi] = Hie/ ■ However, in the case that Rj's are not independent, 
but "mostly" independent (each is only dependent on a small subset of the others), Janson's inequality 
provides a tight bound on Pr[/\.gj Bi]. The following is a short description. 

Let r2 be a finite universal set (a set which contains all elements of interest, including itself) and let A 
be a random subset of Cl such that, for every element r E Vl, Pr[r E A] = Pr, and the events {r E A}ren 
are mutually independent. Let {Ai}i^j be subsets of fi, and J be a finite index set. We define the event 
Ai C A as the event B^. For i,j E I, we write z ~ j if z 7^ j and Ai fl Aj ^ 0; otherwise, i 7^ j. For 
i 7^ j, and i ^ j, B^, Bj are independent events. Further, for i ^ J C /, and i 7^ j, for all j E J, B^ is 



69 

mutually independent of {Bj}j^j, and is thus independent of any Boolean function of those Bj's. 

Let A = P^[Bi A Bj], where the sum is over pairs with (z, j) and (j, i) are counted as one 

pair, and P = Yli^i • following is known as the Janson's inequality. 

Lemma 32: ||27l Chapter 8] Let {B,i}i^j, A, P be defined as above and assume Pr[_Bj] < S, for all 
i G /. Then 

P<Pi[/\Bi]< Pe^/(^-^). 

Now, let A be the union set of events Hi, for all i E I, so that Hi is either Gi, or Gj. Further, let 
Ai be the intersection set of events {Gj}j^NB(,i)^ Bi be the event Ai C A (i.e., Bi occurs if the 
intersection of the events {G'j}jgArg(i) occurs). Then, for i, j E I (i j), and Nq{i) fl Neii) 7^ 0, i ~ j; 
for i j, and i 7^ j, Bi, Bj are independent due to the dependence on two disjoint subsets of {Gk}kei- 
For ijelii< j), and i ~ j, Nsii) = {ku h}, Ne{j) = {kj_i+i, k^, A;^+j_i} (in cyclic order), 
and Neii) U iVeO') = {h, /c^, ke+J^^}. Thus, |iVB(z) U iVeO')! = ^ + J - and 

Pr[S,A5,] = Pr[{ /\ n { /\ G,}] 

= Pr[ A 

k&NB(i)UNBij) 

n 

fcG7Vs(j)UArs(j) 
< g|AfB(*)U7Vs(i)l 
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Thus, 



iel j:i<j<i+e 

< E E 

iel j:i<j<i+e 

= E E 

iei o<j-i<e 

= "E E 

i£l 0<k<i 

= w(e-^)/(l-e)- 

For sufficiently small choice of e, so long as rjq goes to zero, A goes to zero. Thus, in the limit of interest, 
the Janson's inequality results in Pr[/\.g^ 5^] = P. Since Pr[Bi] < 77, P = HiG/ Pi1^] > {l-vY > l-r]q. 

For larger values of e, a block is bad w.p. b.a.b. 77, and hence, an upper bound on the fraction of bad 
blocks is 77. The actual fraction of bad blocks can be shown to be tightly concentrated around the upper 
bound on the expected fraction as follows. 

Theorem 23: For any e > 0, and A > 0, in an OCC with an aperture size a = f2 ((/^/A^)r In ((//Ae)r)), 
and overlap parameter r, over a network of length / with any one-in-one-out worst-case schedule of 
capacity (1 + \)k, for any type of dependency between hyperchunks with negatively dependent neigh- 
borhoods, for any 7a > 0, the fraction of blocks that are not recoverable deviates farther than 7a from 77, 
w.p. b.a.b. e"'^'''', for some positive constant c = 0(A/a). 

Before giving the proof, we state a theorem which is useful in proving concentration results for the 
sum of dependent indicator random variables (see, e.g., [l28|). 

For a given set /, and a sequence of random variables {Xijjg/, a subset J of / is called independent 
if {Xi]i(zj are independent. A sequence {/j} of subsets of / is a cover of /, if Ij = I. A sequence 
{{Ij,Uj)} of pairs {Ij,Uj), where Ij C I and uj G [0, 1] is a fractional cover of /, if J^j-iei^ ^ 1' 
each i E I. A cover or a fractional cover is proper if each set Ij in it is independent. Let M be the size 
(the number of subsets I/s) of the smallest proper cover of /, and Mm be the smallest J^j'^j ^^^^ 
proper fractional covers {{Ij,Uj)}. 

Lemma 33: [|28l Corollary 2.2] Suppose that X = Xlie/^*' i E I, Xi ^ Bemoulli(pj), for 



71 



some < Pi < 1. Let /i = E[X]. For any 7^ > 0, 

Fi[X > (1 + 7„)/i] < e-2^''^'/^-l^l. 

Proof of Theorem [23l' Let / be the set of integers [q] . For every i E I, let Xj be an indicator random 
variable associated with the ith block (Bi), so that X/ is 1 if Bi occurs, and it is 0, otherwise. Then, 
X = XliG/ ^« '•^^ number of bad blocks. Since each block belongs to i contiguous hyperchunks, the 
event of a given block being bad is a function of the events of decodability of the hyperchunks containing 
it. Since for all j E I, jth block belongs to £ contiguous hyperchunks, the next block which does not 
belong to any of these hyperchunks is the (j +i)th block, and so forth (x is chosen properly so that f is a 
divisor of q). Thus, for all j E I, Bj, Bj+£, Bj^g^e are independent. Taking Ij = + ...,j + q — i}, 
for all j E [i], the family {Ij} of subsets of / is the smallest proper cover of /, because for every j E [i], 
Ij is the largest possible subset which contains all the possible indices whose associated indicator random 
variables {Xi}i^j^ are independent. The size M of such a proper cover of / is i. To give an upper bound 
on Mm, let us expand the family {Ij} as follows: the new family U{{Ij}) contains any non-empty subset 
of /j, for each j E [i] (i.e., U{{Ij}) is the union of the powersets of subsets J/s, excluding the null 
set)Cj By definition, U{{Ij}) is a cover. On the other hand, there are 2''^^ — 1 non-empty subsets of Ij. 
By symmetry, each of q/i distinct indices is equally distributed in these subsets, and hence each index 
appears (2^/^ - l)/{q/t) times. Setting ujj = (g/£)/(2«/^ - 1), the inequality Y.j:i(^ij^j > 1 holds with 
equality. Since there are q indices in /, J^jei^J = ^ [(9/0/(2'^/^ - 1)] • Thus, < q [(g/£)/(2«/^ - 1)] . 
Since Pr[i?j] = 77 (assuming the worst-case scenario), the expected number of bad blocks is rjq. For any 
7a > 0, then, from Lemma |33l it follows that 

PT[X>{l + ^a)riq]<e-'", 

by choosing c = 0(r/a£) - (In(fc) - 0{\n{{ai/T)-flr]^))) /k. ■ 
Therefore, for any type of dependency between hyperchunks with negatively dependent neighborhoods, 
w.h.p., for any arbitrarily small constant 7a > 0, the number of unrecoverable message packets is upper 
bounded by (1 + '-)a)f]k. 

^Ihe powerset of a set S is the set of all subsets of S, including the null set and the set S itself. 
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